Friday, October 28, 2011

Evaluating Tech Writer Performance

It's that time of the year again: performance review time.  As I sit here making my way through the stack of  reviews in front of me, I'm reflecting on the process.

I actually like perf reviews.  I have a very strong team.  Perf reviews give me the opportunity to reward the ones who really nailed it this year, and to give others the feedback they need to be one of the writers I'm rewarding at this time next year.

Also, perf reviews give me a chance to celebrate all the great work my team has done over the past year.  Tech writers are invariably on the forefront of the most exciting new technology, and needless to say, there's a lot of fun stuff going on at Google.

Essentially when I'm evaluating tech writer performance, I look at three things:
  • What did the person write?
  • How good was it?
  • How hard was it to write?
What did the person write?
My team comprises about 25 writers, give or take.  That's an extremely lean team when you consider everything we write: http://developer.android.com/, http://code.google.com/http://developers.google.com/, http://www.chromium.org/, and more.  In any given year, most of my writers write the equivalent of at least a couple of books.  They stay busy!  

How good was it?
I read what my team writes, and look for clarity, completeness, and quality.  I also look at stats, such as Analytics data, bug queue metrics, and forum data, that objectively tell me how effective the docs were.

How hard was it to write?
This is the tricky one with tech writing.  You often have to dig to find out what's behind a document.  It might take a day or a month to write the same one-page developer guide; there's really no way of telling on the surface.  It depends on what source information there is to start with, including specs, other docs, or sample code, how complete the API is, how well-designed the API is, and how helpful the related subject matter experts are.  The only way to find this out is to keep in close communication with the writer, and to solicit feedback from the team the writer works with.  

My team has several different levels of tech writers; some straight out of college, some with a long career history under their belt, and many in between.  In general, the more senior the tech writer, the more self-sufficient they should be, and the better they should be able to find creative, strategic solutions and guide the work of others.

How do you think tech writers should be evaluated?  If you work with another role, what differences or similarities are there in how performance is evaluated for that role?

Thursday, October 20, 2011

Top 10 Components of an API Doc

In order of importance:
  1. Sample code
  2. Complete reference, including expected behavior, argument ranges, default values, possible side effects, and related methods
  3. Error handling information
  4. Developer's guide
  5. Changelog or release notes
  6. Installation and configuration information
  7. Tutorial or codelab
  8. Conceptual overview
  9. FAQ
  10. Usage info for related developer tools
What do you think, folks?  What'd I miss?   What do you think is out of order?

Friday, October 14, 2011

Focusing Your Portfolio

Lately, Google's been announcing decisions it's made to focus efforts on key products. I've made similar decisions to focus the efforts of my team, and thought it would be interesting to talk about why and how I did this.

There are a few constant challenges in staffing tech writers:
  • Product teams often don’t think of tech writers until the last minute.
  • It takes time to get to know a developer product well enough to document it, so it’s not usually possible to immediately refocus someone on a different project when requested to do so.  Therefore, staffing takes time.
  • The closer teams get to releasing a product without documentation, the more frantic they get, and correspondingly, the louder they get.
Unfortunately, the squeaky wheel gets a lot more attention than the smooth-running Lexus, so, if you don't have a prioritization framework, you sometimes get roped into making reactive staffing decisions that don’t make much sense in the long run. You may end up with a portfolio of projects whose only connection is that they are staffed by the most insistent product managers.  This may result in your not being able to staff the most important, fastest-moving projects, and losing opportunities to do high-impact work.

My team had fallen into this trap when I stepped up to management.  Google has a myriad of interesting projects going on at any given time.  As a company, it's known for its bottom-up management style. This is great for being empowered to get things done quickly and with a minimum of politics. However, it's historically been difficult to get a straight answer about what projects are most important.

The writers on my team are extremely talented. They do great work on the projects we work on, and their project teams love working with them.  However, previously, our portfolio consisted of a smattering of random projects that, taken as a whole, didn't make much sense.  More importantly, we weren't solving many strategic business problems for Google.  Because of that, we had a tendency to be overlooked when it came time for things like resources, recognition, and headcount allocations.

The first thing I did to resolve this was to step back and make some deliberate decisions about what business problem we were trying to solve.  In our case, that was "Support our developers."  Then I made choices about what kinds of projects would do the most to help our developers.  In some cases these choices led to our working on the documentation for key products.  In other cases we had to think bigger; providing frameworks that let others contribute to the documentation with our guidance, or providing solutions for sticky problems that impacted multiple documentation sets at a time.

Since there wasn't really any clear external prioritization guidance, I had to come up with my own criteria.  To do that, I made some high-level decisions about what factors might indicate that a project is important. The criteria I decided to use to evaluate project requests is as follows:

How many users does the product have?
  • If there is existing documentation, is it one of the X most-read doc sets at the organization based on Analytics data?
  • If the product is an API, how many requests are (or are projected to be) made to the API itself?
  • What’s the (projected) rate of adoption for the product?
How strategic is the project?
  • Is the project in the company high-level goals?
  • Did it get a company or industry award?
  • How many engineers are on the project?
  • How many support people are on the project?
  • If the product is an API, does it enable developers to create applications that work with a strategic company product?
  • Is the project focused on a high-level community goal that the company supports?
How much revenue does the project generate?
  • How much money is the project projected to bring in this year?
  • Will revenue increase or decrease over time?
Now, whenever I receive a documentation request, I ask the project lead to provide data about how the project measures up to these criteria.  I use the data to make unbiased decisions about which projects to work on.  I do this transparently, so that even if people don't always like the decisions I make, at least they understand why I make them.
 
Vetting project requests is a good first step.  The next step is to be proactive about seeking out strategic projects to work on.  I do this by making sure I'm involved in key meetings; especially those where new projects are reviewed and discussed.  This gives me information about what new projects are coming down the pike, in time to plan resources needed to staff them.  At the same time, it gives me the opportunity to meet the project stakeholders and raise awareness of the value of documentation and the work that my team does.  

These days, developer product teams at Google are much more aware of the value of good documentation.  The other day, in a product review meeting, a product manager said "We can have all the great APIs we want, but without good documentation, we have nothing."  (By the way, he's right. :-) ) People think of us when they're planning new projects, and get us involved early.

Focusing our portfolio was not an instant process, to be sure, but it was worth it.  It drastically increased the impact my team's work has on our developers.   Once Google understood the role that tech writers play in supporting our developers, our group gained the status, recognition, and respect that our work merits.

Wednesday, October 5, 2011

How to Hire a Great API Writer

The other day, a friend asked my advice about interview questions for tech writing candidates.  He needed to hire some writers to document some APIs for his company.  Since he isn't a tech writer himself, he wasn't sure what to ask.

The reality is that usually someone other than a tech writer is the hiring manager.  So I thought I'd share my thoughts on how to hire great writers, especially those who write developer documentation.

Screening Resumes  When you screen resumes, look for someone who has experience explaining highly technical information.  If you're hiring a junior writer with little to no documentation experience, look for related roles.  For example, maybe they provided tech support for their school computer lab. Or if they're switching careers, maybe they provided customer support for developers, or worked in an environment testing APIs.  Pro-bono experience writing similar documentation for an open source project is great; it shows proficiency with similar projects, and also shows initiative.

I tend to approach ex-engineers with caution.  Some of them make fantastic tech writers, but it's important to dig deeper into their motivation for a career change.  Look for those who have discovered that they enjoy writing specs and communicating with customers more than they enjoy debugging code, so they've decided to make a career out of writing.  Shy away from people who would prefer an engineering job but, for some reason, can't get hired as an engineer.  At best, they will transfer to a new job at their first opportunity.   At worst, they will hate their job and damage team morale.

In the education section of the resume, look for evidence of continuing education in related topics; for example: technical writing, programming, or project management classes.  A successful technical writer needs to be constantly learning, since the work we're doing is always on the cutting edge of technology, and also because our jobs are so varied and require many skills from many different curricula.

As long as the candidate has a bachelor's degree, don't worry too much about what field it's in.  There are very few technical writing degree programs in the US, and there are as many paths into tech writing as there are tech writers.  I've met successful tech writers who have degrees in journalism, business, computer science, teaching, and philosophy, among others.

Evaluating Work Samples Any tech writer worth his salt will have a portfolio of sample work.  Even someone just getting into the industry should have something to show you: a spec he wrote, an exercise from a class he took, or documentation he wrote for an open source project.

When you request samples, be sure to ask questions to ascertain how much of it the candidate wrote himself.  Did he write it from scratch, revise an existing document, or co-write it?  Did he have support from an editor to help with the writing, or a support from a publisher to help with publishing logistics?  You're really looking for work that the candidate did himself, so ask him for samples he wrote himself.  If he doesn't have any of those available, first ask yourself why not, and then at least get a clear picture of how much involvement the candidate had in writing the samples he does have.

When you review samples, evaluate whether the candidate is explaining things clearly.  If you don't have any relevant domain knowledge, it's possible that you won't understand everything; however, at least part of it should resonate.  Also, look for typos, misspellings, grammatical errors, and other things that indicate potential laziness.  Tech writers' work isn't always perfect, but writing samples should be as close to perfect as possible.

Conducting the Interview During the interview, try to ascertain that the candidate:
  • knows what goes into good technical documentation
  • has the technical aptitude to pick up the particulars of your project
  • shows the initiative needed to create great documentation without minimal impact on the productivity of others in your company
To that end, here are some basic interview questions to ask, as well as advice on what kinds of answers to look for.

Q: What kind of information should be included in an API reference?
A: In addition to the basics (explanation of what the method does, arguments, and return values) a good writer should mention one or more of the following: acceptable ranges of values for arguments, default values, what happens if the method gets called using the default values, possible side effects of calling the method, a list of related methods that are often called with the one in question.  Developers read the documentation to find out information they can't get by simply looking at the header file, and they'll be frustrated if that information isn't included.

Q: What kinds of documents need to be written to document an API?
A: At minimum, you'll need a reference, a basic developer's guide that explains how you put the methods together, and a basic sample application.  If possible, the reference should be automatically generated using javadoc, doxygen, or similar technology.  If possible, the sample application should be compilable, so someone can just use it as the starting point for their own application.  If time allows, developer documentation should also include as many of the following additional components as possible: A tutorial that walks through a complex sample app, one concept at a time, release notes that explain what changed between releases, so developers know what they have to do to update their application to the latest version, and in-depth documentation that explains any of the concepts that the API relies on; especially if the developer isn't likely to be familiar with those concepts.

Q: Explain the technology involved with one of the samples you sent.
A: Even you aren't the target audience for the sample, the candidate should be able to explain it in a way that you understand it.  A key skill for a tech writer is being able to explain things clearly.  If you don't understand it after the candidate's explanation, it's not you--it's the candidate.  Move on.

Q: Now assume you're talking to your grandmother.  Explain the technology to her.
A: A good tech writer can explain even the most complex of concepts simply to a non-technical audience.  I once had a candidate explain the concept of inheritance by saying that his DVD player, stereo and TiVo have an inheritance relationship, because they are all different manifestations of the same kind of technology.

Q: Explain the process you went through to write one of the samples you sent.
A: You're looking for evidence of self-sufficiency.  The candidate should do all the research and reading she can (within reason) to avoid wasting engineers' time with basic questions.  The candidate should not rely on a spec, or any other document where someone else is writing the majority of the documentation.  With enough research, and asking the right types of questions, a good tech writer should be self-sufficient enough to write documentation herself.  Chances are you're looking for someone who functions as a one-woman show, so in an ideal situation, the candidate publishes  documentation herself, rather than passing source docs off to a publishing team or webmaster.  If the writer used an editor, find out what kind of edits were recommended.  If a developmental (structural) edit was required, then that demonstrates a fundamental inability to organize information logically, and you should walk away.  Look for the ability to take and incorporate feedback.  Ask how the candidate requests information and feedback from engineers, and look for evidence of the writer taking responsibility for the content, rather than pushing that responsibility off to the reviewers.

Q: Explain what's happening in a simple snippet of code (that you provide)  This code should be written in one of the languages with which the candidate professes familiarity.  It should contain a couple of basic constructs, like a loop, and an if, then directive.  It should have an explicit return value and variables, and may also modify inherited variables.  This is one of the few times where it's better to have less comments, so that you can ensure the candidate is really reading the code.
A: The candidate should be able to basically explain what the code is doing.  She should ask good questions about things that aren't obvious.  Give the candidate time to think before answering.

What Not to Ask: Don't ask a tech writer candidate questions about how to optimize code efficiency,  questions about design patterns, questions about how to resolve bugs in code, or even puzzle/logic questions.  Almost universally, none of the skills touched on by those questions are relevant to a tech writing job.  There are very few tech writers who could answer these sorts of questions.

Checking references Finally, it's important to check references. You'll probably get more information if you talk to someone who has left the company; today's HR laws dictate that any reference request made to a current employee will probably get passed directly to HR, who will be able to tell you little more than the dates that the candidate worked there.

Ask open-ended questions, such as:
  • What are candidate's strengths?
  • What are candidate's weak spots?
  • What's it like to work with candidate on a project?
  • If the reference worked with the candidate on a project he discussed during the interview, ask the reference for his perspective on how that project went.
  • Would you hire candidate again?
Thus concludes my advice for hiring great technical writers.   Good luck!  Let us know how it goes.

Oh, and if you're a tech writer looking for job-hunting advice, stay tuned for follow-up posts on how to get hired as a tech writer, and how to break into tech writing.

Monday, September 26, 2011

Soliciting Useful Community Collaboration (or, how to get good community content that isn't interspersed with Viagra ads)

Community collaboration is key, for the reasons I've mentioned in previous posts.  But getting good collaboration isn't straightforward.  You have to architect the site correctly, accommodating the reality of different collaboration types.  You also have to guide the dynamics of a community in order to expose the useful feedback and prevent the entire site from being overrun by trolls, bad information, and Viagra commercials.

There are a couple of different models for encouraging community feedback.

Letting people add notes to the doc

One model is to maintain a canonical, source controlled doc, but let people add notes to the end of each doc page.  php.net is an example of this model can work.  As illustrated by the page I linked to, what makes this work is that contributors post useful information at the end of each page.  The community-contributed content adds to what is already in the documentation, and is helpful to the wider community.

One advantage of this model is that it opens the docs up to collaboration, while ensuring the quality and reliability of  the canonical docs.  This offers protection against trolls, and also against people who have good intentions, but are just wrong.

The disadvantage of this model is that it usually garners a different type of feedback than you'd want. Users generally treat the comments section at the end of a page as either a forum, or as a place to give feedback about the docs.  At best, community feedback generally consists of something like, "I'm trying to do X, please help" or "This page states that this method returns A, but it actually returns B.  Please fix the doc."  At worst, the feedback areas end up being a dumping ground for spammy links to online pharmacies and porn sites.

The way you get around this is to heavily moderate contributed comments, and address and remove those that aren't appropriate.  Over time, the hope is that the docs will get enough appropriate comments that people will use those as an example, and begin to post the kind of thing you're looking for.  Regardless, this will always require some moderation effort.

In my opinion, your best bet with this model is to bypass it altogether.  Instead, link to a dedicated forum where people can garner answers from the community, and link to an issue tracker where people can report issues with the docs.  If that's what people want to do, it's better to just accept it and provide appropriate tools to help.

Letting people modify anything in the doc

A second model is wiki, which opens up the docs entirely to community collaboration.  The community can modify anything at all.  developer.mozilla.org is an example of this model. You can access the collaboration history of each page by selecting "This page -> History" in the upper right hand corner.  The history for the page I linked shows extensive community collaboration.  Wikipedia is another well-known example of this model.

Usually, contributors are required to sign in before they post.  Contributions can be reviewed by a central moderator, or by the community, or both. 

Generally, as a contributor gains more credibility over time, their posts are subject to less oversight.  Often what you end up with is a set of trusted core contributors, who are enthusiastic and familiar with the technology.  You also have a long-tail of one-time or occasional contributors.

I think this model is where the documentation industry is going, and it's generally a good model to adopt.  There are a few things you need to do to make it successful, though.

First and foremost, recognize the value of your top contributors.  Give them some way to earn a public reputation on the site.  Include them in events (either as attendees or speakers), send them t-shirts, give them early access to your APIs.  As Pamela Fox mentioned in her post "How to Alienate a Community 101", though, you have to be careful that you do this in a way that recognizes the spectrum of contributions.  Pamela's post contains good information about how to do that.

Next, adopt a good model for moderating content.  The most sustainable way to do this long-term is to let the community moderate.  Have a voting system, and use votes to flag content that needs to be reviewed.  Allow access level based on reputation, so those users who are most helpful in the forums have the most leeway to modify the docs.  Even if you do these things, though, recognize that you'll still need to put some work into moderating content in the short term.

Mindtouch, and Confluence are a couple of out-of-the-box solutions for this model.  I haven't worked with either one so can't comment on their effectiveness.

Encouraging, and linking to, discussion in a forum

Embrace the fact that much of the useful hands-on discussion occurs in forums. There are several good sites for this, but one of the best sites I know of is Stack Overflow. It is already in wide use, for good reason.  It supports reputations and badges, which makes for a good contributor experience.  It supports tags, which is useful for finding and linking to relevant content. 

If conversation about your product is happening in one of these forums, link to it.  If not, encourage conversation there. 

Friday, September 23, 2011

Speaking to your audience

I've been having trouble getting started on this blog.  There are a lot of things I'd like to talk about here... developer support, API usability, managing a team of writers, and, of course, documentation.  The problem is that I haven't been able to figure out what to talk about first.

It just struck me that the reason for dragging my feet is that I haven't thought about who my audience is.

When you start a new tech writing project, the first thing you do is figure out your audience.  Then you figure out what they're trying to do. Once you know your audience's starting point, and figure out where they're trying to go, you can figure out what you need to do to help them get there.

The challenge with blogs, and with public web documentation in general, is that absolutely anyone can read what you're writing.  If you're reading this right now, you could be a fellow tech writer (either just getting started in your career, or a veteran), a developer, or a tech writing manager.  And those are just the basics.  Heck, you could be a fly fisherman who stumbled across this page accidentally (and is probably wondering why he's still here.)

Web docs have similar challenges, especially those for APIs that are offered freely to the general public.  One developer might be a student, who is trying to piece together an app over cold pizza at 3am because he saw an interesting article about the technology on Slashdot.  Another developer might be writing an enterprise app as part of her day job.  A third developer might be working on an cutting-edge application for that startup he always dreamed about creating. 

One developer might create web apps every day, and know everything that's out there about it.  Someone else might be used to writing drivers in C, but have zero experience with Java.  And yet another person might have no development experience whatsoever; they're trying to get something working by copy-and-pasting sample code and hoping for the best.

How do you write one document that works for everyone?

Believe it or not, it is possible, depending on your definition of "write one document." You can't do it if you expect all of your readers to sit down and read a thousand-page manual from start to finish.  But if you use good navigation and information architecture, and well-configured search, then anyone can find what they need.

The navigation of your docs should work for as many of your typical developers as possible.  People should be able to quickly find the docs that explain what they're trying to do, without having to read through a lot of irrelevant prose.  As a writer, you should think about how to reuse information intelligently.  For example, everyone will need to authenticate their app, so the docs that explain authentication should be linked from lots of places.  On the other hand, only certain apps will need to use particular sequences of methods.

The web is a goldmine of information.  There's no need to explain the basics, such as how REST works.  Find a good resource that explains it, and link to that resource.

But most importantly, just take the time to think about who your audience is.  Being cognizant of who is reading will help guide what you write, even if you just tuck that knowledge away in your subconscious and don't do anything about it.
So back to this blog.  If you're new to tech writing, or new to tech writing management, I hope this blog is helpful to you.  Please feel free to email me with questions and topic requests.  I'll link to additional helpful resources as I find them.  If you're an old pro, or a developer, let's get a conversation going.  Please jump in and argue my points, if you're so inclined! Regardless of who you are, you can navigate the blog by topic or by search.

Lots of heads are always better than one

With the advent of web-based docs, users completely gave up the notion of reading an entire 1000-page docset from start to finish.  Here's how it works now:
  • People only read documentation under duress.  If people want light reading, they'll pick up the comics section of the local paper.  If people want to learn how to use an API, they'll find some sample code (probably on a forum somewhere) and try to make it work for their own app.  They won't touch the official docs until they get stuck, and can't easily find the answer to their problem elsewhere.
  • The next thing they'll do is search the official docs for the answer and/or scan the nav bar for something that might help.  If they find something that seems to be relevant, great!  If not, you've already lost them.
  • If the section they navigate to has the answer to their problem, they might be inclined to keep reading.  If not, you've lost them.
  • If they consistently find useful information in the docs, they'll start to trust the docs and will keep coming back to them.  But, as one of my favorite candidates, Doug Anderson, said to me in an interview, "You only get so many opportunities with documentation before you teach people not to look at it."
And that's why it's so hard for a single tech writer to create engaging, satisfying documentation.  There are too many ways it can go wrong.

That's also why the traditional definition of one writer being solely responsible for a thousand-page manual no longer works.  One writer can't scale to support an entire community of diverse developers.  She can't predict all of the different types of code they'll be trying to write, and can't anticipate all the ways they'll get stuck.  So we have to change the way we approach documentation.

One of the most important roles of the official documentation is to introduce libraries, features, and tools for the first time.  When there is no other existing documentation, people will read your docs.  That will always be true.

After people have noodled on your docs for a while, and figured out how to do what they're trying to do, they'll become a strong, capable, worldwide team, skilled at helping specific slices of audiences do exactly what they're trying to do. That's the power of the community.

And that's where our jobs as tech writers change.  We're writing docs, to be sure, but we also need to help the community support itself.  Let people help each other do tasks similar to the work they're doing themselves.  Cultivate a community of helpful developers, and give them easy ways to share their knowledge.  Make sure that helpful pages, responses, and code snippets are surfaced in search results, give pointers to the best ones in the docs, and give credit where credit is due.  Because lots of heads are always better than one.