Log in

Will Woods, Fedora Testing Guy [entries|archive|friends|userinfo]
Will Woods, Fedora Testing Guy

[ website | https://ohjeezlinux.wordpress.com/ ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

There's something stuck in my brain. [May. 12th, 2009|06:33 pm]
Will Woods, Fedora Testing Guy
Saw this on boingboing earlier and I can't stop thinking about it:
1. No one cares about what you think, unless you do what you think.
2. No one cares what you do, unless you think about what you do.
3. No one ever really cares what you say.

Spending less time on talk - and more effort on thinking things through and getting them done - seems like some good advice for me right now.
linkpost comment

Why glxgears is slower with Kernel Modesetting (and why it doesn't matter) [Mar. 13th, 2009|04:43 pm]
Will Woods, Fedora Testing Guy
or: "glxgears is not a 3d benchmark"

One interesting fact came out of yesterday's Intel KMS Test Day. Everyone noticed that glxgears is much slower under KMS/DRI2 than it was before (e.g. in Fedora 9 or 10). A typical result was ~1000FPS in F10, but only ~440FPS in Rawhide. So what's changed?

Here's the deal, as best as I understand it (thanks to krh/halfline/ajax for trying to explain it to me): glxgears is rendering an insanely simple scene - so simple that the actual 3D rendering time is basically zero. So the only thing glxgears really tests is the performance of glXSwapBuffers() - basically, how fast we can push render buffers into the card. This operation is slower with DRI2, but - roughly speaking - unless it was an order of magnitude slower (e.g. glxgears drops from 1000FPS to under 100FPS) it wouldn't make any real difference.

So if you're going to be comparing "3D performance" - please, don't bother with glxgears. ajax has suggested a couple of things that might make better benchmarks:
  • The mesa-demos package has a few useful things - teapot in particular is nicely simple.
  • sierpinski3d and glblur from xscreensaver-gl-extras also work well.
  • extremetuxracer is, in my opinion, the most fun way to benchmark 3D.

If you see major (100% or more) performance differences between KMS and non-KMS in those apps, it's probably worth investigating further.
link6 comments|post comment

.i586 packages: not a big deal [Mar. 9th, 2009|01:58 pm]
Will Woods, Fedora Testing Guy
A note about .i586 packages in Fedora 11, and why you shouldn't worry about them:

Since we have a new compiler, some RPM checksum improvements, and - for x86 users - changes to the default architecture support, we've done a mass rebuild of every package in Fedora. x86 users will likely notice the following changes:

  • All .i386 packages have been rebuilt for i586.

  • Instead of building four kernel packages (i386, i586, i686, and i686-PAE) we're now building just two kernels: i586 and i686-PAE.

    • Newly-installed x86 systems that have 'pae' and 'nx' in /proc/cpuinfo will get i686-PAE.

    • Everything else will use kernel.i586.

Here's the thing: moving to i586 isn't a big deal because we're already optimizing for modern (i.e. Pentium 4 or later) processors. For the past couple years we've been using gcc's -mtune=generic flag, which optimizes for "the processors that were most common when that version of GCC was released" (according to the gcc info page). The only thing that moving to i586 really changes is the instruction set that's used. i586 adds a handful of new instructions over i386 - notably CMPXCHG8B, which is useful for efficient atomic operations - but nothing world-changing. This probably isn't going to give you more FPS in glxgears.

As for the kernel, the only difference between i586 and i686 is the CMOV instruction family, which - as Linus explains here - generally performs worse than the equivalent i586 instructions on modern CPUs. So moving the kernel from i686 to i586 should have no real effect on performance.

I'm sure there will be much discussion on the subject, but I'd like to see some good benchmarks - real data, rather than people reporting that Fedora 11 is "snappier" because it's "Pentium optimized", or "slower" because "the kernel is only i586 now".
link1 comment|post comment

PING WITH DATA [Feb. 10th, 2009|11:17 am]
Will Woods, Fedora Testing Guy

If you want to ask a question on IRC, NEVER DO THIS:

<someguy> wwoods: ping

What's gonna happen if I'm not there? An hour later when I get back to IRC, all I know is - you wanted something. I have no idea what. So then all I can do is ping you back, and if you're not around, then we have to keep sending pings back and forth until we're both online at the same time.

An empty "ping" is the same as saying "Can I ask a question?" You just did. Just ask the question.

<someguy> wwoods: ping - what's that blue thing doing there?

Now when I get back to IRC, I can actually answer your question immediately. And if I'm online you'll get your answer right then - no need to spend the extra network roundtrip on a useless ping/pong.

Ping with data - ask the question, mention what you wanted to talk about, whatever. Just don't send empty pings. It's a waste of time.

This has been an Unwelcome IRC Etiquette™ lesson
© 2009 Surly Fedora QA Guy Enterprises, Inc. All wrongs reversed.

linkpost comment

python-bugzilla - give me your queries! [Aug. 19th, 2008|05:12 pm]
Will Woods, Fedora Testing Guy

I've been working on a branch of python-bugzilla that supports multiple different Bugzilla variants. The current version (0.3) only supports the functions provided by Red Hat's Bugzilla 2.18.x. This branch (which will become python-bugzilla 0.4) adds support for the upstream 3.0 and 3.2 WebServices and Red Hat's shiny new 3.2 instance. (Hooray!)

Yes, I'm sure you've noticed that we've updated to Bugzilla 3.2. Besides the shiny new web frontend there's also new and shiny XMLRPC code for python-bugzilla to use. It still supports the old RHBugzilla stuff, and it now supports all the upstream Bugzilla 3.2 services, but there are some additional new-style methods that will someday make their way to upstream Bugzilla - although probably not until 4.0.

The point is this - I'm trying to get python-bugzilla using the new methods wherever possible. This includes a new and slightly different Bug.search() call, and I want to make sure I can make the switch without breaking everyone's existing code.

So here's my request. If you use python-bugzilla, send me your queries - or just a link to the code, if that's too much to ask - and I will do my best to make sure that your code will work, unchanged, with the new upstream(ish) Bugzilla methods.

I'll update soon to talk about what else is new in python-bugzilla 0.4, but having some real-world test code to work with would be a great boon. Thanks!

linkpost comment

Writing Test Plans [Jul. 22nd, 2008|01:35 pm]
Will Woods, Fedora Testing Guy

Everyone knows what Fedora QA does. "They're the testers. They test stuff." Sure, that's true. But.. what, exactly, do we test? And how?

That's where Test Plans come in. A Test Plan is a document that tells the testers how to test something.

A good software Test Plan should at least answer these four questions:
  1. What special hardware / data / etc. is needed to test this (if any)?
  2. How do you prepare your system to test this? What packages need to be installed, config files edited, etc.?
  3. What specific actions do you perform to check that the software is working like it's supposed to?
  4. What are the expected results of those actions?

Your answers can be short: "get a bluetooth keyboard and yum install bluez-gnome newer than 0.25" answers question 1 and 2 just fine. But they need to be complete and explicit: "Get an appropriate keyboard and install the new packages" is not helpful.

Questions 3 and 4 are answered by Test Cases. A Test Case is a single, specific thing to test - a list of actions to perform, and the expected results of those actions. Depending on how complicated a package or feature is, you could have one Test Case or a whole bunch of them.

While writing a Test Plan, pretend that you have an intern whose job it is to test Fedora releases. This brave, brave soul has a few years' experience as a Fedora user and basic familiarity with using the commandline to configure stuff, install packages, and so on.

Your job is to tell this person how they can test your favorite package or your cool new feature, so they can either a) tell their friends about how great it is, or b) tell you (via bugzilla) if it breaks.

Here's an example Test Case for Banshee, originally written by our own Balaji Gurudass:

    Test loading a Local File in Banshee.
  1. Start Banshee
  2. Choose "Import Media" from the "Media" menu
  3. Select "Local Files" from the import source dropdown and click the "Import Media Source" button
  4. Select the music file and click Open
  5. Find the song in the music library and play it.
Expected Results:
  1. Banshee starts successfully
  2. The "Import Media" menu item brings up the "Import Media to Library" dialog
  3. The chosen music file is imported into the Banshee library
  4. The music file can be played and the tracker moves along and shows time elapsed.
This gives us a clear, easy-to-follow way for anyone to prove that a new banshee package does (or doesn't) work like it's supposed to. Or, at least, one small part of it.

So there's an overview of how to write test plans. Currently we're keeping Test Plans in the wiki. There's a section in each Feature Page for a Test Plan, and if you wanted to write a Test Plan for a specific package (or set of packages), you could add a wiki page named "QA/[package] Test Plan". For example, here's QA/Banshee Test Plan.

Next time, I'll talk about why I think this stuff is so important.

link2 comments|post comment

preupgrade is not 1.0 [May. 19th, 2008|05:23 pm]
Will Woods, Fedora Testing Guy
So, yes, preupgrade is in the F8 repos now, and that's nice. But it's not 100% complete and that's why it's not in the menus or anything yet.

The current version - 0.9.3-3 - will fail to boot the installer if your /boot directory is a) on RAID1 (#444497), or b) on your root partition (#446826). These should both be fixed in the next preupgrade release.

At the moment, though, there's a bug that prevents preupgrade from being able to finish any upgrades without a network connection. This is the bug that Greg hit during his preupgrade. Luckily, as he noticed, you can bring up the network in the installer, but you can't use anything fancy - no wireless, no VPN, etc.

This may not be fixable without a respin of Fedora 9 (or, at least, a custom stage2.img specifically for preupgrade). I'm not really happy about it.

Hopefully we can do this right for F10. Sigh.
link5 comments|post comment

Translation Halp! [Apr. 1st, 2008|06:20 pm]
Will Woods, Fedora Testing Guy
So if you've used GNOME in Rawhide lately you probably noticed this new "About this Computer" thing in your System menu:

That was me! I did that!

The problem is.. those strings need to be translated into all the various languages I don't speak.

Eventually the patches involved will be in upstream GNOME or some Fedora-specific package. And then we can use the magic of Transifex to do the translation properly. But while that's being sorted out, I still need to get those strings translated for F9.

If you can help, please check out this wiki page and add translations for your language.

Thanks a million!
link3 comments|post comment

We're intercontinental when we eat french toast. [Nov. 25th, 2007|12:28 pm]
Will Woods, Fedora Testing Guy
So today (as with most days) I hit http://planet.fedoraproject.org/ to see what's been going on in the Fedora Community.

The first six posts, in order, were in Spanish, Kashmiri (well, ok, English, but talking about Kashmiri), Chinese, Indonesian (I think?), French, and English.

Fedora might technically be US-based but it's pretty obvious that we are a worldwide community. And that's really cool. It makes us monolingual* Americans feel kind of silly, though.

* Well, OK, I took 6 years of French in high school and 2 years of Japanese in college, but I haven't spoken either language in years.
link1 comment|post comment

Fedora 8 - RC3 vs. Final bits [Nov. 4th, 2007|10:24 am]
Will Woods, Fedora Testing Guy
Someone asked in my last post if RC3 was the final version.

The answer is: Nope - we had to do two respins after RC3.

Thursday night we found, filed, debugged, and fixed the fsck-at-boot / rhgb hang at first boot problems (see bugs #357401 and #362721 respectively), completed the fix for #357401 (dmraid+livecd) and then rebuilt all the install and Live images to create RC4 for Friday morning. Friday's rawhide report notes these changes.

During Friday we found and fixed a bug in selinux-policy-targeted where restorecon etc. would screw up at install time and leave you with SELinux non-functional. Once again Jesse rebuilt all install/Live images - creating RC5 - with this change included. This change is listed in Saturday's Rawhide Report.

Once the RC5 bits were built (mid-afternoon Friday) we desperately tried to smoketest them and get them rsync'd to the Raleigh office before 9pm so that we could hand them off to the internal IT guys. (After 9, it would be too late to get it done for Monday, and we'd have to delay the release.)

The bits landed at 8:45pm. All our testing showed that the bugs above were, indeed, fixed. Jesse made the call, got a response, and that was that.

There were heroic efforts from a lot of people in those last two days, and I raise my glass to one and all.
link8 comments|post comment

[ viewing | 10 entries back ]
[ go | earlier/later ]