To guess is cheap. To guess wrong is expensive.
— Chinese proverb
"[…]The danger of using any [salary] survey is that you might end up with the wrong numbers. Behind every survey is a set of assumptions and criteria that, if not made sense of, can lead one to compare apples with oranges, or computer programmers with program directors." The following questions are designed to help you determine how much value to place on the data you see in the various salary surveys. They are excerpted from the article State-of-the-art compensation by Jerry Useem, published in Inc Magazine in October 1998. Please read each question and answer completely so you can understand why each must be considered.
To this list that Inc provided, I’m going to add one more courtesy of Mary-Ellen Mort of JobStar:
- Are the companies polled comparable in size to your own?
- Big companies, you might have heard, tend to pay more than small ones do. But the size of that disparity tends to be proportional to a job’s elevation on the organizational chart: while a chief financial officer will earn a whole lot more at a big company than at a small one, two data-entry clerks at similarly sized companies will likely make about the same.
- Is the geographic focus appropriate?
- Here, the inverse is true: geography matters little for high-level positions and a lot for low-level ones. The reason? The job market for managers and professionals is national, while the market for blue-collar workers is regional, or even local.
- How were the jobs "matched"?
- This is perhaps the key question. A study is of little value if the jobs in question aren’t comparable with one another, and simply matching them by title doesn’t do the trick–especially in this era of hybrid workers and job titles such as "chief evangelist." (For those reasons, many surveyors are switching to so-called maturity studies, which measure a worker’s skill set rather than a job description.) So find out: was the matching process conducted by mail or by in-depth, face-to-face interviews?
- What statistics were used?
- Many studies report a simple average salary, when in fact a median figure–representing the midpoint of the range–would be more appropriate. Typically, the median is 3% to 5% lower than the average.
- How old are the data?
- In rapidly moving fields such as information technology, where salaries for certain jobs can leap by 50% within a few months, fresh information is of the essence. What’s important is not when the survey was published but when the salaries in question were put into effect.
- How many companies were polled?
- Experts say that a survey isn’t credible unless it includes at least 20 organizations and discloses a full list of them in its report.
- Who was included in this survey?.
- If I said "the average employee at XYZStartup makes $100,000/year," you might be impressed. However, if I then tell you that the survey included all XYZStartup employees from the entry-level mailroom sorter up to the CEO, that alters the data considerably.