Friday, October 28, 2011

Evaluating Tech Writer Performance

It's that time of the year again: performance review time.  As I sit here making my way through the stack of  reviews in front of me, I'm reflecting on the process.

I actually like perf reviews.  I have a very strong team.  Perf reviews give me the opportunity to reward the ones who really nailed it this year, and to give others the feedback they need to be one of the writers I'm rewarding at this time next year.

Also, perf reviews give me a chance to celebrate all the great work my team has done over the past year.  Tech writers are invariably on the forefront of the most exciting new technology, and needless to say, there's a lot of fun stuff going on at Google.

Essentially when I'm evaluating tech writer performance, I look at three things:
  • What did the person write?
  • How good was it?
  • How hard was it to write?
What did the person write?
My team comprises about 25 writers, give or take.  That's an extremely lean team when you consider everything we write: http://developer.android.com/, http://code.google.com/http://developers.google.com/, http://www.chromium.org/, and more.  In any given year, most of my writers write the equivalent of at least a couple of books.  They stay busy!  

How good was it?
I read what my team writes, and look for clarity, completeness, and quality.  I also look at stats, such as Analytics data, bug queue metrics, and forum data, that objectively tell me how effective the docs were.

How hard was it to write?
This is the tricky one with tech writing.  You often have to dig to find out what's behind a document.  It might take a day or a month to write the same one-page developer guide; there's really no way of telling on the surface.  It depends on what source information there is to start with, including specs, other docs, or sample code, how complete the API is, how well-designed the API is, and how helpful the related subject matter experts are.  The only way to find this out is to keep in close communication with the writer, and to solicit feedback from the team the writer works with.  

My team has several different levels of tech writers; some straight out of college, some with a long career history under their belt, and many in between.  In general, the more senior the tech writer, the more self-sufficient they should be, and the better they should be able to find creative, strategic solutions and guide the work of others.

How do you think tech writers should be evaluated?  If you work with another role, what differences or similarities are there in how performance is evaluated for that role?

Thursday, October 20, 2011

Top 10 Components of an API Doc

In order of importance:
  1. Sample code
  2. Complete reference, including expected behavior, argument ranges, default values, possible side effects, and related methods
  3. Error handling information
  4. Developer's guide
  5. Changelog or release notes
  6. Installation and configuration information
  7. Tutorial or codelab
  8. Conceptual overview
  9. FAQ
  10. Usage info for related developer tools
What do you think, folks?  What'd I miss?   What do you think is out of order?

Friday, October 14, 2011

Focusing Your Portfolio

Lately, Google's been announcing decisions it's made to focus efforts on key products. I've made similar decisions to focus the efforts of my team, and thought it would be interesting to talk about why and how I did this.

There are a few constant challenges in staffing tech writers:
  • Product teams often don’t think of tech writers until the last minute.
  • It takes time to get to know a developer product well enough to document it, so it’s not usually possible to immediately refocus someone on a different project when requested to do so.  Therefore, staffing takes time.
  • The closer teams get to releasing a product without documentation, the more frantic they get, and correspondingly, the louder they get.
Unfortunately, the squeaky wheel gets a lot more attention than the smooth-running Lexus, so, if you don't have a prioritization framework, you sometimes get roped into making reactive staffing decisions that don’t make much sense in the long run. You may end up with a portfolio of projects whose only connection is that they are staffed by the most insistent product managers.  This may result in your not being able to staff the most important, fastest-moving projects, and losing opportunities to do high-impact work.

My team had fallen into this trap when I stepped up to management.  Google has a myriad of interesting projects going on at any given time.  As a company, it's known for its bottom-up management style. This is great for being empowered to get things done quickly and with a minimum of politics. However, it's historically been difficult to get a straight answer about what projects are most important.

The writers on my team are extremely talented. They do great work on the projects we work on, and their project teams love working with them.  However, previously, our portfolio consisted of a smattering of random projects that, taken as a whole, didn't make much sense.  More importantly, we weren't solving many strategic business problems for Google.  Because of that, we had a tendency to be overlooked when it came time for things like resources, recognition, and headcount allocations.

The first thing I did to resolve this was to step back and make some deliberate decisions about what business problem we were trying to solve.  In our case, that was "Support our developers."  Then I made choices about what kinds of projects would do the most to help our developers.  In some cases these choices led to our working on the documentation for key products.  In other cases we had to think bigger; providing frameworks that let others contribute to the documentation with our guidance, or providing solutions for sticky problems that impacted multiple documentation sets at a time.

Since there wasn't really any clear external prioritization guidance, I had to come up with my own criteria.  To do that, I made some high-level decisions about what factors might indicate that a project is important. The criteria I decided to use to evaluate project requests is as follows:

How many users does the product have?
  • If there is existing documentation, is it one of the X most-read doc sets at the organization based on Analytics data?
  • If the product is an API, how many requests are (or are projected to be) made to the API itself?
  • What’s the (projected) rate of adoption for the product?
How strategic is the project?
  • Is the project in the company high-level goals?
  • Did it get a company or industry award?
  • How many engineers are on the project?
  • How many support people are on the project?
  • If the product is an API, does it enable developers to create applications that work with a strategic company product?
  • Is the project focused on a high-level community goal that the company supports?
How much revenue does the project generate?
  • How much money is the project projected to bring in this year?
  • Will revenue increase or decrease over time?
Now, whenever I receive a documentation request, I ask the project lead to provide data about how the project measures up to these criteria.  I use the data to make unbiased decisions about which projects to work on.  I do this transparently, so that even if people don't always like the decisions I make, at least they understand why I make them.
 
Vetting project requests is a good first step.  The next step is to be proactive about seeking out strategic projects to work on.  I do this by making sure I'm involved in key meetings; especially those where new projects are reviewed and discussed.  This gives me information about what new projects are coming down the pike, in time to plan resources needed to staff them.  At the same time, it gives me the opportunity to meet the project stakeholders and raise awareness of the value of documentation and the work that my team does.  

These days, developer product teams at Google are much more aware of the value of good documentation.  The other day, in a product review meeting, a product manager said "We can have all the great APIs we want, but without good documentation, we have nothing."  (By the way, he's right. :-) ) People think of us when they're planning new projects, and get us involved early.

Focusing our portfolio was not an instant process, to be sure, but it was worth it.  It drastically increased the impact my team's work has on our developers.   Once Google understood the role that tech writers play in supporting our developers, our group gained the status, recognition, and respect that our work merits.

Wednesday, October 5, 2011

How to Hire a Great API Writer

The other day, a friend asked my advice about interview questions for tech writing candidates.  He needed to hire some writers to document some APIs for his company.  Since he isn't a tech writer himself, he wasn't sure what to ask.

The reality is that usually someone other than a tech writer is the hiring manager.  So I thought I'd share my thoughts on how to hire great writers, especially those who write developer documentation.

Screening Resumes  When you screen resumes, look for someone who has experience explaining highly technical information.  If you're hiring a junior writer with little to no documentation experience, look for related roles.  For example, maybe they provided tech support for their school computer lab. Or if they're switching careers, maybe they provided customer support for developers, or worked in an environment testing APIs.  Pro-bono experience writing similar documentation for an open source project is great; it shows proficiency with similar projects, and also shows initiative.

I tend to approach ex-engineers with caution.  Some of them make fantastic tech writers, but it's important to dig deeper into their motivation for a career change.  Look for those who have discovered that they enjoy writing specs and communicating with customers more than they enjoy debugging code, so they've decided to make a career out of writing.  Shy away from people who would prefer an engineering job but, for some reason, can't get hired as an engineer.  At best, they will transfer to a new job at their first opportunity.   At worst, they will hate their job and damage team morale.

In the education section of the resume, look for evidence of continuing education in related topics; for example: technical writing, programming, or project management classes.  A successful technical writer needs to be constantly learning, since the work we're doing is always on the cutting edge of technology, and also because our jobs are so varied and require many skills from many different curricula.

As long as the candidate has a bachelor's degree, don't worry too much about what field it's in.  There are very few technical writing degree programs in the US, and there are as many paths into tech writing as there are tech writers.  I've met successful tech writers who have degrees in journalism, business, computer science, teaching, and philosophy, among others.

Evaluating Work Samples Any tech writer worth his salt will have a portfolio of sample work.  Even someone just getting into the industry should have something to show you: a spec he wrote, an exercise from a class he took, or documentation he wrote for an open source project.

When you request samples, be sure to ask questions to ascertain how much of it the candidate wrote himself.  Did he write it from scratch, revise an existing document, or co-write it?  Did he have support from an editor to help with the writing, or a support from a publisher to help with publishing logistics?  You're really looking for work that the candidate did himself, so ask him for samples he wrote himself.  If he doesn't have any of those available, first ask yourself why not, and then at least get a clear picture of how much involvement the candidate had in writing the samples he does have.

When you review samples, evaluate whether the candidate is explaining things clearly.  If you don't have any relevant domain knowledge, it's possible that you won't understand everything; however, at least part of it should resonate.  Also, look for typos, misspellings, grammatical errors, and other things that indicate potential laziness.  Tech writers' work isn't always perfect, but writing samples should be as close to perfect as possible.

Conducting the Interview During the interview, try to ascertain that the candidate:
  • knows what goes into good technical documentation
  • has the technical aptitude to pick up the particulars of your project
  • shows the initiative needed to create great documentation without minimal impact on the productivity of others in your company
To that end, here are some basic interview questions to ask, as well as advice on what kinds of answers to look for.

Q: What kind of information should be included in an API reference?
A: In addition to the basics (explanation of what the method does, arguments, and return values) a good writer should mention one or more of the following: acceptable ranges of values for arguments, default values, what happens if the method gets called using the default values, possible side effects of calling the method, a list of related methods that are often called with the one in question.  Developers read the documentation to find out information they can't get by simply looking at the header file, and they'll be frustrated if that information isn't included.

Q: What kinds of documents need to be written to document an API?
A: At minimum, you'll need a reference, a basic developer's guide that explains how you put the methods together, and a basic sample application.  If possible, the reference should be automatically generated using javadoc, doxygen, or similar technology.  If possible, the sample application should be compilable, so someone can just use it as the starting point for their own application.  If time allows, developer documentation should also include as many of the following additional components as possible: A tutorial that walks through a complex sample app, one concept at a time, release notes that explain what changed between releases, so developers know what they have to do to update their application to the latest version, and in-depth documentation that explains any of the concepts that the API relies on; especially if the developer isn't likely to be familiar with those concepts.

Q: Explain the technology involved with one of the samples you sent.
A: Even you aren't the target audience for the sample, the candidate should be able to explain it in a way that you understand it.  A key skill for a tech writer is being able to explain things clearly.  If you don't understand it after the candidate's explanation, it's not you--it's the candidate.  Move on.

Q: Now assume you're talking to your grandmother.  Explain the technology to her.
A: A good tech writer can explain even the most complex of concepts simply to a non-technical audience.  I once had a candidate explain the concept of inheritance by saying that his DVD player, stereo and TiVo have an inheritance relationship, because they are all different manifestations of the same kind of technology.

Q: Explain the process you went through to write one of the samples you sent.
A: You're looking for evidence of self-sufficiency.  The candidate should do all the research and reading she can (within reason) to avoid wasting engineers' time with basic questions.  The candidate should not rely on a spec, or any other document where someone else is writing the majority of the documentation.  With enough research, and asking the right types of questions, a good tech writer should be self-sufficient enough to write documentation herself.  Chances are you're looking for someone who functions as a one-woman show, so in an ideal situation, the candidate publishes  documentation herself, rather than passing source docs off to a publishing team or webmaster.  If the writer used an editor, find out what kind of edits were recommended.  If a developmental (structural) edit was required, then that demonstrates a fundamental inability to organize information logically, and you should walk away.  Look for the ability to take and incorporate feedback.  Ask how the candidate requests information and feedback from engineers, and look for evidence of the writer taking responsibility for the content, rather than pushing that responsibility off to the reviewers.

Q: Explain what's happening in a simple snippet of code (that you provide)  This code should be written in one of the languages with which the candidate professes familiarity.  It should contain a couple of basic constructs, like a loop, and an if, then directive.  It should have an explicit return value and variables, and may also modify inherited variables.  This is one of the few times where it's better to have less comments, so that you can ensure the candidate is really reading the code.
A: The candidate should be able to basically explain what the code is doing.  She should ask good questions about things that aren't obvious.  Give the candidate time to think before answering.

What Not to Ask: Don't ask a tech writer candidate questions about how to optimize code efficiency,  questions about design patterns, questions about how to resolve bugs in code, or even puzzle/logic questions.  Almost universally, none of the skills touched on by those questions are relevant to a tech writing job.  There are very few tech writers who could answer these sorts of questions.

Checking references Finally, it's important to check references. You'll probably get more information if you talk to someone who has left the company; today's HR laws dictate that any reference request made to a current employee will probably get passed directly to HR, who will be able to tell you little more than the dates that the candidate worked there.

Ask open-ended questions, such as:
  • What are candidate's strengths?
  • What are candidate's weak spots?
  • What's it like to work with candidate on a project?
  • If the reference worked with the candidate on a project he discussed during the interview, ask the reference for his perspective on how that project went.
  • Would you hire candidate again?
Thus concludes my advice for hiring great technical writers.   Good luck!  Let us know how it goes.

Oh, and if you're a tech writer looking for job-hunting advice, stay tuned for follow-up posts on how to get hired as a tech writer, and how to break into tech writing.