Friday, December 12, 2008

Open Positions at Netflix

I have an immediate need for a software engineer in test for the Customer Account group on the Web QA team at Netflix.

I have an upcoming need for a software engineer in test for the Software Infrastructure group on the Web QA team at Netflix.

I also have an upcoming need for a build and release engineer.

Any help from anyone that happens to read this message in a bottle would be most welcome.

For all of them, I need someone who has a proven ability to:
• get the right things done when there's always too much to do
• solve problems in a proactive and elegant way
• communicate clearly and succinctly

First, a description of what I'm looking for in the Software Engineer in Test -- Customer Account


Netflix has a goal of 20 million members by 2012. Part of keeping those members happy is helping them manage their own data and efficiently sharing that data with other parts of Netflix. Your team provides services to members, internal Netflix teams and external partners. These services span authentication, account information management, and authoritative answers to questions about member data.

We are in the process of consolidating our business logic into services and SDKs from where it lives today: the presentation tier and stored procedures in the database. As we untangle and streamline our code, we need a safety net of automated tests. The tools and automated tests that you build will help insure that planned changes don't cause unintended consequences. What you build will help prevent defects by finding them in the test environment and detect defects that make it out to production.

At Netflix, you will work with a team of intelligent and capable software engineers who are committed to delivering high quality software through test driven development, test automation and a deep partnership with quality assurance.


• Expert in assessing software quality in an ecommerce environment using:
- Test driven development
- Java, Tomcat, JDBC and Apache
- jUnit test automation (or equivalent)
- Web services
• Tests as part of cross-functional teams and supports back-end application interfaces
• Understands relational databases and SQL deeply
• Develops and maintains production system and data monitoring tools
• Desires to work in a fast paced, evolving, growing, dynamic environment

Now, here's the description for Software Engineer in Test -- Software Infrastructure


The number of people getting their movie entertainment through the Internet is continuing to grow at a rapid pace. Between DVDs by mail and Instant Watching, Netflix wants most of those subscribers to be Netflix members. Part of keeping those current and future members happy is making sure our systems are built from components and services that will scale to handle the volume.

The Software Infrastructure team builds components that include:
• management of Queue and rental history data
• a Netflix server reference implementation
• security APIs and services
• session management APIs and services
• message bus services for logging and data transfer
• explorations into The Cloud for all of the above

You will be a key part of the core team that ensures our broader development teams are using an intuitive and robust set of services to build the website and operations applications. You will ensure that we capture and accurately deliver data, and that we do it in a timely fashion. You will ensure that our internal services meet the needs of the developers consuming them. At Netflix, you will work with a team of intelligent and capable software engineers who are committed to test driven development, test automation and a deep partnership with quality assurance.


Expert in understanding, assessing and providing feedback on the quality of high traffic, n-tier, web based environments including:
• Test driven development using Java, Tomcat, JDBC/Oracle, service buses and Apache with jUnit or TestNG test automation (or equivalent)
• Cross-functional testing of back-end system interfaces

Experienced with testing:
• Traditional and non-traditional e-commerce applications
• High traffic/high availability components
• Service oriented architectures

Deep understanding of:
• relational databases and SQL
• Linux or some other unix flavor
• a scripting language like Perl or Python

Familiarity with:
• testing under open source frameworks, such as Struts, Hibernate, Spring, etc.

Desire to work in a fast paced, evolving, growing, dynamic environment

Finally, I bring you a description for what I'd like to see in a Configuration and Release Engineer:


Web Engineering at Netflix provides our public face to our customers, so it's very important that we deliver an easy to use and engaging web experience. As part of the Web QA team, you'll be responsible for the team's test servers and their environment. Other duties include herding the changes to the code base from development, through testing and out to production. You will coordinate the change process across all of the teams involved, so you need not only technical chops, but also excellent communication skills. This position is part Linux admin, part software engineer and part project manager. The ideal candidate should be equally comfortable owning bash scripts, java code and ant configuration files, and will have an insatiable craving for making the deployment process more and more push-button and more and more secure.

• Excellent communication and inter-personal skills
• Working experience in web based environments utilizing:
- Java, Tomcat, Apache and other web applications
- Perforce and other source control applications
- Ant and the 'x'Unit family of test automation applications
• Skilled in Unix shell and SQL
• Desire to work in a fast paced, evolving, growing, dynamic environment

• Perl hacking
• Bugzilla maintenance and modification
• Configure and use Kintana (Mercury IT Governance Center) deployment application
• Build and manage automated build/test/deployment environments

Saturday, December 06, 2008

What is Applesauce the Gateway To?

In the last few weekends of September, I was making applesauce from the Fuji and Granny Smith apple trees we planted a couple of years ago. I had just picked the last apples, except for one lonely Fuji. At the time, I wasn't sure if it would ever ripen to the point where it would come off easily when one of us happens to be there to test it. It might have just dropped off in the middle of the night, and wind up in the ant's larder instead of ours. As it was, it did come off just before Thanksgiving.

Back in September, I had just finished putting a batch of cooked applesauce into a couple of 2 quart containers and left them on the counter-top, loosely covered. I wanted them to cool a bit before sticking them in the refrigerator. It was very deeply satisfying to pick and wash and peel and core and dice and simmer. Sampling the apples at different stages along the way lit up paths in my brain leading to very comfortable and comforting places. The snap and explosion of biting into a freshly picked and washed apple is like summer fireworks. Peeling the apples, I felt the juice slip over my fingers, making the peeler and the apple slick. I wondered just how much of the juice went down the drain instead of into the sauce. I've asked Jen to put a peeler on my Christmas list and I wonder if it will make next year's applesauce any different.

Lucy had introduced me to chewing some of the peel just after it drops into the sink. You get the tartness of the skin without the moderating sweetness of much juice or flesh. It's work, chewing on the peel, but it recalls other pleasures that are satisfying because they are work and not pure pleasure. Running is like that for me as well: it's hard work, but I feel better afterward.

Coring the apples is always an adventure. Attempting to pop out the heart of the apple with the seeds while minimizing the degree of flesh that goes with them. I ponder whether an apple corer will have a better or worse core/flesh ratio, since it's an indiscriminate cylinder versus the custom "cone plus wedges" that I'm cutting out.

If you compare the dollar cost of a quart of organic applesauce at Trader Joe's to the time and effort of growing, waiting for and harvesting the apples, then processing them all the way into sauce, it seems penny wise and pound foolish to spend the time. However, like many other things in life, the main value comes down to the doing, and not the monetary value of the physical product.

Software As A Manufacturing Activity

Another book that I read recently was Goldratt's The Goal. It's gotten me thinking about software development within the context of manufacturing. If a software development cycle is constrained by one of the intermediate steps, it either will starve the downstream processes or let software of inferior quality out in front of the paying customers. Neither is a desirable outcome.

So, you have two problems from the world of moving around atoms that may be applicable to the world of moving around bits. First, the problem of coordination and scheduling that supply chain management gurus have thought about. Second, the problem of throughput and productivity that constraint based manufacturing theory has gone a long way towards optimizing.

How much can be applied directly, and how much just doesn't apply because turning out software doesn't suffer from the same scarcities as turning out physical products?

I guess that's why I'm trying to think and write about it.

Experience Chain Management

You heard about it here first, folks. I've just created a new discipline: Experience Chain Management.

It was fueled mostly by some reading I did of Lou Carbone's writing, in his book Clued In: How to Keep Customers Coming Back Again and Again.

The core of it is that every business is trying to create a valuable customer experience to keep the money coming in the door. However, nobody thinks about the internal consumers of the business processes that deliver those experiences to the paying customers. Someone upstream can sabotage the effectiveness of the downstream team members and, without even realizing it, impair the customer experience.

Simple example that I'm periodically wrestling with:

  • Developer publishes a code change past the start of the testing cycle
  • The QA engineer doesn't regress that part of the code because they don't know about the change
  • A bad bug slips through to production
  • A customer can't get what they want done, and gets angry instead

I'm thinking that there are probably some lessons to be learned from all of the work done on supply chain management. Even though you're shuffling bits instead of atoms, when you have a team turning out software that drives a customer experience, it's still an awful lot like manufacturing. You have:

  • raw materials (product ideas)
  • manufacturing (coding)
  • assembly & testing (build, configure, QA)
  • delivery (deployment & operations)

Of course, the thing I need to think about more is how downstream components and processes can block or impair the productivity of upstream partners. For that, you need to look at a different discipline (see next post).