Thursday, December 14, 2006

Senator Tim Johnson Conspiracy Theories

Well, I just checked google, and there aren't any conspiracy theories that have yet surfaced about Senator Johnson. I certainly wish him well, both for his own sake, and for the sake of the country.

Just seems to me that we should check him for any Polonium that George W. Putin, or one of his minions, left lying around.

Wednesday, December 06, 2006

Employment opportunities redux

Here's an excerpt from an email I sent to a friend who was telling me that she didn't want to commute quite as far as Los Gatos for a job:
Anyway, if you *were* working, and there was anyone that you'd want to have on your team again, let me know who they are. They're not necessarily even traditional QA people. Anyone who is smart, curious and diligent, and who understands how computers work qualifies.

I'm trying to build a different brand of QA. It's the java engineers who are ultimately responsible for the correctness, completeness and scalability of their product -- you can't test in any of those attributes. All QA can do is assess whether those attributes exist and advise the developers on any significant lack.

I'm not looking for someone who knows silktest or loadrunner or even jUnit inside and out. I want someone who can understand the fact that our business is driven by three things: 1. customers can sign up, 2. customers can add movies to their queue, 3. customers get discs shipped to them

Everything else is just syntactic sugar that dresses up our core semantics.

So, as I've mentioned before, I'm looking for a couple of people who are interested in addressing the issue of software quality at Netflix, and I'm NOT looking for a browser-poker or someone who dreams up test cases for an offshore outfit to throw warm bodies at.

If you want to help us run with scissors, apply

Tuesday, December 05, 2006

Free Blockbuster Rentals for Netflix Customers

On HackingNetflix there is an article about being able to take the address flap from your Netflix rental envelope into a Blockbuster store and getting a free rental. I wonder if BBI is harvesting the addresses for targeted direct mail campaigns? That free movie might not only be costing you the gas to drive there, but it might cost you a bunch of future junk mail, too.

Thursday, November 30, 2006

Employment opportunities

Most people think of Netflix as either a movie company, or a logisitics company, because on the surface, we appear to buy DVDs from the studios and just mail a bunch of plastic around the country.

However, the secret sauce to the business is that we're really a software company. We're solving problems in how to scale the website to serve 20 million members (Java, Oracle). We're working on machine learning problems (like the Netflix Prize) and scaling problems with our recommendations engine (C++, mysql). I'm most closely involvd in figuring out how quality fits in to all of these development processes. I'm changing the role of a QA engineer from someone who pokes at a browser to someone who writes software to test software.

Then there's all the stuff that the back end teams do (logisitics, inventory, order generation). Once all the software's figured out, it has to run somewhere, and we need good system admins, DBAs and managers for those people that can plan for growth and install, configure and deploy to that plan.

The key thing is that we aren't doing this with an army of developers like an eBay, Yahoo or Google. We've got a relatively small team of really smart software engineers solving problems in an efficient and clever manner. A very deeply ingrained cultural value here is to only hire very senior people who understand how to solve problems, and then go and solve them. It's how we've been able to scale from just over 1MM customers when I joined 3.5 years ago to nearly 6MM customers today. The team that builds the website hasn't grown by more than 10-15% over that time, and we've added a ton of new stuff to the site. It's all because we don't scale the business by doing what we were doing just with more people; we scale through clever automation.

There's a list of what's open at our jobs page, but if there's something that anyone's specifically interested in, let me know. We're always looking for the right kind of person more than we're looking for a particular technology or skill.

Tuesday, August 29, 2006

I stuck the equivalent of two dead bodies in the ground today

That would be the six bags of concrete that weighed sixty pounds each. I'm building an emergency shed out in back of the house. This has been a bit of a saga, going on for the better part of eight months. Maybe I'll tell that whole tale some other time. For now, I can share the more 'concrete' part of the story.

If you haven't worked with concrete then you may be underestimating, to a great degree, my latest foe. Six bags of portland cement, sand and small rocks. Each bag is 0.45 cubic feet of concrete. A half a cubic foot sounds pretty small. It is pretty small when you're trying to do a footing around the perimeter of a 8'x10' shed foundation. I did six bags on Sunday and six bags today while waiting for an air conditioning repairman to show up. It only took me an hour to mix and distribute the six bags, but that was the longest stinking hour of my life. Add on top of that the effort to get the 360 pounds (hence, the two dead bodies reference) of dry concrete down from the garage to the shed site, and I didn't have to go running this morning to get a workout. I considered whether to cut down the amount of water I was mixing in, to balance out the amount of sweat dripping off me that was falling into the mixing trough.

I actually have about 4 more bags to go, then I can actually start assembling the shed on the flat, level foundation that I've built. I'll try to snap some pictures so you all can mock my efforts.

Friday, August 25, 2006

You looking for a job? I'm looking for someone to work!

As you may or may not recall, I'm the QA manager for the Netflix website. I've also been given the responsibility to manage our customer service call center application. If you fit the following description and also happen to like the idea of the job, drop me a line or even better, apply directly.


Senior Quality Assurance Engineer — Customer Service Automation

The Position:
Join Netflix and help insure that every member who has a problem gets that problem solved quickly and effectively. Working on our customer call center software, you will help the people that help our members.

You will provide your expertise in understanding what to test, as well as how to test. Given a schedule that is always too short and a feature list that is always too long, you'll identify and efficiently evaluate those changes to our software that pose the greatest risk to the smooth and effective operation of the customer service staff.

We are looking for an experienced engineer with a background testing applications involving .NET, Oracle, SQL, PL/SQL, telephony integration, email and customer service application user interfaces.

• Validate and verify the correct operation of the customer service application
• Continuously improve and extend the automated regression test suite

You are:
• An advocate for the customer service representatives with regard to usable system design
• An advocate for customer service management with regard to accuracy and completeness of the system
• A bridge between the development organization and the operational groups supporting the system

• 5+ years of rich GUI client application testing, building and deployment
• Deep experience with Oracle and PL/SQL or equivalent database systems
• Experience with C# and nUnit development and testing
• Experience with testing high-traffic, highly scaling, multi-threaded software design
• Able to work autonomously and flexible enough to shift priorities based on changing needs of the business
• Familiarity with use of .NET Remoting, Web Services, 3rd party .NET controls, TestComplete, Parasoft .TEST and Tuvox are all plusses
• A Bachelors or Masters in Computer Science or equivalent engineering

Harder's getting easier every day

A few days ago, I wrote about running, and how sometimes it wasn't physically difficult, but mentally difficult to put my body in an active state. I guess sometimes if you can distract the mind, the body will do what it has to.

Case in point, today while running, I was tangling with a statistical problem. There are performance monitoring tools built in to the Netflix website server that keep track of how long it takes to do stuff along the way to presenting a page to a member. We call them tracers. We keep track of the average execution time, how many times it's executed and also buckets for ranges of execution time. For example, we keep track of how many times it took less than 10 milliseconds, and how many times it took 10-50 milliseconds, 50-100 milliseconds, etc. Using them, we can diagnose the health of the system and discover if a recent change is the source of site problems.

I am studying whether we can learn if the system is healthy from comparing the skew of that distribution of buckets. I'd worked out a database query that gave me a value for skew, but I was measuring the skew of the number of hits per bucket, instead of the skew of all the values of all the individual hits (which is what I actually want). For example, if the eight buckets have the values of (10, 14, 100, 20, 3, 0, 0, 0), I would a value based on those eight numbers, with a relatively high positive skew of 2.57, because the 100 is a pretty far outlier to the right, making this distribution of eight numbers 'pointy' on the positive end. What I was really after is the skew of the 147 individual data points with 10 instances of value 1, 14 of value 2, 100 for value 3, etc. This would have a much more neutral skew of -0.56 indicating that it's slightly pointy on the left.

Anyway, while pondering all of this, I was running along and didn't even have time to whine to myself about not wanting to be out there running, and it was over before I knew it. I think I made pretty good time today, too.

It's kind of bugging me that I lost track of another thought that I had running. Something to do with just a little bit of peril. Now I can't recall what it was. I guess maybe I need the inverse of an iPod. Something I can run with and dictate the flotsam that crosses my mind, and be able to play it back later.

Tuesday, August 22, 2006

Interesting page from ten years ago

I saw an interesting puzzle with a surprising answer while tripping around the net today looking for something else (isn't that always the way)?

The twist to the story entertained me, and the commentary following the twist moved me. Even though the commentary is going on ten years old, most parts are relevant or even more relevant today.

Tuesday, August 15, 2006

Easy's Getting Harder Every Day

So I run several times a week. Usually every other day. I go running in Rancho San Antonio near my home. Usually just under three miles. Some uphill and some downhill and some flats. Several weeks back, I was running and thought about what it would take to run across America. After all, Interstate 80 runs from San Francisco all the way to New York City. 3000 miles at three miles a day. A thousand days. Just about three years. I could do that, easy. Be kind of hard to only do three miles a day, though. I get through my three miles in 20-30 minutes, usually. I could probably slow down a bit and do 30 miles a day over the course of 10 hours of stops and starts. That gets it down to 100 days. Start in San Francisco in April with nothing but a running outfit and a credit card, then run and walk the whole day. Go to sleep whereever you're at and do it again the next day. Might be able to make it to Manhattan for the Fourth of July.

That seemed doable a few weeks ago (even if a little crazy). Lately, though, it's been harder to get up the hills, and even going down the hills has been awkward and sluggish. Now, running across the country seems not only crazy, but impossible. The hardest part of exercise is willfully putting myself into a state of discomfort. I know that I'll feel good when I'm done, but overcoming that inertia is the biggest obstacle. Once I'm going and I'm away from the house, then it's easier to go, but those first several scores of steps are hard.

Thursday, July 13, 2006

I wish I could write

I want to write the perfect paragraph. I want it to enfold the sum of all human desire. I want it to motivate the world to do well by others. I want it to have rhythm and grace. I want it to enrapture all readers and purge them of ill intentions. I want honesty and freedom and the ability for all to reach the pinnacle of who they can be. I want to believe in the best of all possible worlds, and I want to believe that the world we live in is that world.

Wednesday, July 12, 2006

Googling me

So, I decided to type in my name "Rob Fagen" in the Google search box to see what comes up. Here it is:

The ADB Drinking game (me)
"DBs in Vegas" (me)
Google Groups: (me)
Membership Directory (not me)
About STLAHA (not me)
[PDF] Gambler (me)
Radio Days--and Nights (me)
Council defers appointment to Planning Commission -- Los Altos ... (me)
The First Tee of The Seacoast Home Page (not me)

Not bad... only three out of ten misses, and the top three were me. Interesting that the oldest stuff is ranked the highest. Not one mention of this blog, though. I guess maybe after this entry gets indexed that I might see more of me.

But it's good for you, isn't it?

The hardest part for me about running, or any exercise for that matter, is willfully putting myself in a state of discomfort. If there was a magical machine that could stress my muscles for me, then I don't think I'd mind the puffing, sweating and thumping heart pain. It might violate the Geneva Convention, but it would be easier to exercise if I were able to abdicate responsibility for continuing.

I guess that's part of what makes a top-tier athlete: the ability to push past limits and train yourself to go past limits that stop everyone else.

That's probably what makes for a top-tier anything.

Thursday, July 06, 2006

Sudoku Combat

The Sudoku Combat blog discusses the change history of a really good online Sudoku challenge game. I like it. A little quirky, but it gets the game on with some competition (which always adds a little spice!).

Sunday, June 18, 2006

Well, so much for that.

Hrmph. So I'm sit down to play in the Pokerstars blogger championship. First hand of the tournament, and I get the big blind. First bad sign. I get 83 off-suit. Second bad sign. There are a couple of callers, and I check. Flop comes K83 rainbow. I check. Late position caller makes a pot sized bet, everyone folds between him and me, and I raise to 450. He comes back over the top, all-in. I have him on a King for sure, and anything but an 8 or a 3. However, I'm in for a rude awakening when he turns over K8 off-suit. I somehow managed to not find one of the other two 3's in the deck. I wished him good-luck and went on my way. I guess I may have been overaggressive in calling the all-in. Still sucks.

Thursday, June 15, 2006

Maybe another shot at the Big Dance?

Texas Holdem Poker

I have registered to play in the PokerStars World Blogger Championship of Online Poker!

This Online Poker Tournament is a No Limit Texas Holdem event exclusive to Bloggers.

Registration code: 2521058

Thursday, June 08, 2006

That $100 Laptop Again

There is a place where you can pledge to purchase one of the OLPC laptops for $300 if 99,999 other people also pledge to do so by Oct 31, 2006. With that $300 purchase, you'd be getting one of the laptops for yourself and funding two more for the children elsewhere who are the focus of the program.

I'd recommend pledging. It was featured on Slashdot a week or two ago and got a big spike in pledges, but it's tapered back off. Projected pledges by the deadline are about 7,000.

Friday, May 05, 2006

Given Up

So, I've given up on the Full Tilt freeroll series. The play is just too crazy to continue investing the time. For the 27 events I entered, I had a best finish of 134 out of 1800. My worst finish was 1714th. You may recall that a seat plug averaged 269th.

My average finish was 646, with a stdev of 392. My median finish was 581. If my finishes were normally distributed, I should have had a cumulative chance of finishing 36th or better of 5.96%, but I never made good on it.

After the 9th attempt, I started keeping track of my finishing hands. Of the 18 where I kept track, I got my money in with far the best of it 9 times (things like AA cracked by KK, or AA vs AxQc and four clubs came).

They play being uniformly horrendous didn't really help me as much as I thought it might. It kind of worked against me in that no matter how many times I went in with a 2:1 or 3:1 or 14:1 advantage, there's be some nutcase still out there that would draw out on me.

Thursday, March 16, 2006

Back in the saddle again

FullTiltPoker is running a promotion consisting of twice daily freerolls of up to 1800 entrants, where the top 36 from each event roll up into a weekly freeroll of about 1000 entrants, of which the top 9 from each week in a given month join frequent players and other good customers of the service in a tourney where the top 20 are given $10k seats for the big dance. They're giving away 20 seats per month for the next five months.

I found the combination of:

  1. free
  2. often
  3. convenient
to be too much to not obsess over.

So, I discussed with my lovely wife whether she would divorce me and/or kick me out of the ouse if I decided that I wanted to try and win a seat through this series. This in mind of the fact that I've already taken my yearly foray at the Oaks, as earlier reported. This in mind of the fact that she and I had previously discussed my annual foray being executed on-line, instead of in a cardroom, and that was roundly rejected as the temptation and availability of playing online for money was too disquieting to her.

In preparation for this discussion, I ran an experiment. I installed the client and registered for one of the daily tourneys and then just left the seat unoccupied to be blinded away, just to see how long it lasts. My hypothesis being that with the insanity of free play, I wanted to see how long it took for the 'real' tourney to begin. Turns out that of the 1800, generally 1300 of them are gone in the first hour. My test subject lasted until 283rd place.

That's a lot of crazy and/or dead money in there, I decided.

Armed with these facts, Jen & I had our discussion, and she actually agreed to a trial run. A big part of it is that they dailys generally start after the kids are in bed and are running when we are usually just watching TV or reading in bed, anyway.

So, I get to play in the dailys until:

  1. I win a seat
  2. I'm too grumpy in the morning to take care of the kids

I've played in four of them so far, and have not finished higher than the low 400's. Three of these times I went in with the best of it, and the most recent time was ended with a total brain fart with a double gutter on the flop to which I made the idiot end of a straight. I was actually on my A-game up to that point and had about a 4x average stack. I am not as upset and I could be about the brain fart, as up to that point, I'd identified and avoided stupid situations I'd fallen for in the past, as well as having identified and exploited obvious weaknesses in other players to take pots.

So, for now, I'm in and I'm stoked. I still have faint hopes for the Oaks freeroll, but now I've got a second chance.

I figure that given the competition, I'm about a 500:1 against getting a seat. With each of the three rounds taking about 5 hours from start to finish, it could be as little as 15 hours to get a $10k seat. An additional part of their promotion is if you play on FTP money and get to the final table, you get $50k extra prize money, $100k for top three and $10MM if you win the event. The $10MM is $1MM/year for 10 years as part of an endorsement contract, where you have to play 10 hrs/week on FTP and promise to wear only their t-shirts.

Anyway, hopefully see you at the WSOP :)

Friday, March 10, 2006

I Heart Lemurs

Apparently so does this guy. A rather comprehensive collection of everything you'd ever want to know about lemurs of all types. Also some good pictures of the critters, like rruffed02.jpg and ringtailed07.jpg

Originally, I posted direct links to the pictures, but the host appears to be blocking access from external referrers. Maybe I can cut & post them here...

ruffed lemur twins
ringtailed lemurs eating

Monday, March 06, 2006

Not Visiting South Dakota Any Time Soon

I was disheartened to read a story about S.D. passing a law making abortion illegal, even when the mother was raped or if the fetus is endangering her life.

This seems reactionary to the point of irrationality. There were, of course, the required quotes from Operation Rescue. It made me think, how come there isn't an 'Operation Choice.'

Instead of Operation Rescue people blocking access to clinics and acting in an intimidating way towards women choosing to exercise control over their reproductive functions, Operation Choice people would provide emotional support and a calm environment within which a woman could make a considered and informed decision about a very important thing going on in her body.

I didn't mean to be a politician

So I was recently up for consideration to be part of Los Altos, CA's Planning Commission. I actually live in the County of Santa Clara, not within the city limits of Los Altos proper. I do have a Los Altos postal mail address, and do consider Los Altos to be the town I live in. However, this fine distinction became an issue, apparently.

I was in the past involved with the Bernal Heights East Slope Design Review Board when living in San Francisco, and I enjoyed the task of keeping an eye on the neighborhood architecturally, and providing a conduit for compromise between the needs of the existing neighbors and the needs of those wanting to improve their properties. So, when I saw an ad inviting applications for the Los Altos Planning Commission, I tossed my hat in the ring.

The original city council election almost four weeks ago resulted in a couple of congratulatory emails and welcomes to the commission. However, the next day, I heard from the City that there had actually been a tie in the voting that wasn't detected until after the meeting, and that there would be a run-off at the next council meeting. I thought it was no big deal, and raised the question about city vs county at that time. I was told that my address had been cleared through the registrar of voters, so I let the issue slip from my mind.

Between then and the next council meeting, I met with the one city councilperson who wasn't able to be at the original interview, and we seemed to get on quite well regarding the planning commission's job and what my role would be on it. We also discussed in passing where I live. It once again didn't seem to be an issue.

I came to find out that it actually is a very important issue. I guess I'll read the city council minutes when they get published to find out the details, or maybe I'll read the article in the next edition of the Town Crier, since I got an email from a reporter and spoke with her briefly on this subject.

Tuesday, February 07, 2006

How Can I Hire QA Engineers That Write Software?

Software is an interesting animal. It can do useful things like help you find movies you love. Software can also test other software to help insure that movie-lovers get the best possible movie-finding experience. I manage the quality assurance group for the Netflix website. For over a year, we've been aggressively pushing towards more test driven development. We've constructed a jUnit based automated framework for testing that lowers the barriers to unit test creation for the development engineers. We've integrated the EMMA code coverage tool with our nightly regression test runs to get a handle on how our level of testing is improving. We've developed code that allows us to execute tests within the context of our Tomcat application, relieving some of the burden of fixture and mock-object creation. What I'm having a much harder time doing is finding QA engineers that can wear both hats:
  1. can analyze the underlying business and insure that the developed system accurately follows that model
  2. can effectively develop software that automatically tests the website and underlying tiers in a relevant and productive way
I've looked on craigslist, LinkedIn and plumbed the depths of our recruiting database looking for this rare bird. Anyone out there have a good idea of where else to look?

Sunday, January 29, 2006

The Story of My Life

Random bit of video that hits close to home on some points and just made me laugh on other points. Nicely produced mockumentary on the whole Dungeons & Dragons nerd theme.

Friday, January 27, 2006

Hamas & Israel

On NPR this morning, I heard a comment by an Israeli official that he would not sit down to negotiations with a Palestinian government led by Hamas. His statement was something like -- What are we going to negotiate, the means of our own destruction?

Part of Hamas' platform is the destruction of the state of Israel. This is clearly not a good way to make friends and influence people.

Perhaps Israel should not categorically refuse to sit down at the table with a Hamas led Palestinian Authority. Perhaps the answer to the question "What are we going to negotiate" has an obvious answer. Perhaps the first item on the agenda is the removal of the 'destruction' plank from Hamas' platform.

But I suppose it's easier for a diplomat to dig in his heels than to work towards a reasonable compromise. I can't say that I'm completely surprised. It's hard to work with someone who wants you dead, even if the best option for a first step is to get him to stop wanting you dead.

Herb Greenberg needs better listening skills

Let's look at what Greenberg is criticizing Netflix for:

Herbism #1 --
"So what if the "beat" was the result of lower general and administrative expenses?"
It would seem to me that the ability to improve the financial efficiency of a business by reducing unneccesary expense would be a good thing.

Herbism #2 --
"So what if higher margins for the quarter were largely the result of lower content costs?"
My assessment would be very positive if Netflix can leverage its increasing subscriber base to better negotiate with content providers, and can use its technology to steer customers to lower cost content that still makes the customer happy.

Herbism #3 & 4 --
"So what if average revenue per subscriber fell, hitting an all-time low, while subscription acquisition costs were up -- hitting an all-time high? So what if the company raised revenue and subscriber guidance while guidance for pre-tax earnings remained unchanged?"
Maybe Herb didn't listen to the conference call when Reed said that Netflix is going to first hit $50MM in pretax earnings and then grow that at 50% per year, then, everything else goes into marketing to accellerate growth, but SAC will never go above the lifetime subscriber value? Sounds like growing earnings to me. Sounds like accellerated subscriber growth without forcing economic losses on the business. Seems pretty efficient and well managed to me

Tuesday, January 24, 2006

Where I Work

It's kind of odd, because my new work address isn't yet in Google Maps. So, I got into the neighborhood and then painted in a splotch where there isn't yet a road. Enjoy.

Monday, January 23, 2006

Step one, get a million dollars...

So, over the weekend, I made my annual foray into the world of Bay Area tournament poker. I played at The Oaks Club in Emeryville in a No-Limit Hold'em tournament that is part of a series leading to a freeroll in June that will have as its first prize a $10k seat in the Big Dance. I've posted previously on last year's adventure.

I met my father-in-law there to guide him through his maiden voyage into the "wretched hive of scum and villany" that is Mos Eisley... err... Bay Area tournament poker. I got there a bit before noon and bought us both in to make sure last year's scheduling snafu didn't get repeated. Then we went off to lunch at a pretty good cafe within walking distance. I had the chicken & waffle, and he had the corned beef hash. Both tasty. We talked about strategy and the mechanics of the game in a cardroom, and in tournaments specifically.

We returned in time for the show, and sat down to play with 107 other hopefuls. I got one or two playable hands, and so at the end of the first three periods, had built my 1k starting stake into a whopping 1,240 (they use 20's instead of 25's at The Oaks). I rebought, and learned that F-I-L had rebought out of necessity a little bit earlier, but he had won a hand since then, and so had a bit over the 1,500 he got for the rebuy.

After the break, I got a couple hands, and in the next two rounds had built my stack up to a bit over 10k. Then I went cold. I didn't drag a pot until we got down to around 36 people left, and I was down to 2100 with 200 antes and 1000/500 blinds. Not looking good for the home team, but I woke up in middle position with QQ and it held up. Quadrupling through helped my attitude, even though I was still short stacked. From there, I built up to a peak of about 40-50k before busting out in 6th place. This gave me 5 points in the freeroll race. I figure I'm a longshot for playing again in June, but it could happen if the freeroll point distribution is heavily enough weighted towards a smaller group of regular tournament players. Of course, in the two weekly events played prior to yesterday, there were no repeaters. More news as it happens, if it happens.

F-I-L said he had a very good time, but it was exhausting. He managed to come in 25th, where 20 places were paid. An excellent showing considering that all he had was book larnin'.