Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Download
Standard view
Full view
of .
Look up keyword
Like this
1Activity
0 of .
Results for:
No results containing your search query
P. 1
How Can I Create My Favorite State Ranking?: The Hidden Pitfalls of Statistical Indexes

How Can I Create My Favorite State Ranking?: The Hidden Pitfalls of Statistical Indexes

Ratings: (0)|Views: 4|Likes:
State economic rankings, though popular with think tanks, policymakers, media and even some academics, cannot be taken at face value. Such rankings typically are subjective and provide little meaningful information, despite their pervasiveness.

For example, the Small Business and Entrepreneurship Council's Small Business Survival Index – one of more than a dozen state rankings touted across the country – is based on factors such as taxes, health care costs, minimum wage level and the presence of right-to-work laws. However, this measures government costs without factoring in benefits generated by government, such as infrastructure and small business development centers.

Kauffman Foundation research has been used as the basis for ranking states, most notably on the rate of entrepreneurial activity, formalized in the Kauffman Index of Entrepreneurial Activity. This is an index based on data from the U.S. Census Bureau and Bureau of Labor Statistics, and it has never intended to be used to claim that one state or another is "better" for entrepreneurs.

Forthcoming Kauffman research shows that many rankings fail even to correlate with company owners' business climate perceptions. Further, the proliferation of state rankings can lead policymakers and economic development consultants to misuse them, either celebrating a conveniently positive ranking or initiating efforts to address a poor ranking, when neither action is based on valid economic indicators.

The authors conducted a series of exercises to demonstrate how such rankings can be manipulated. The exercises were based on eight state-level indicators related to innovation and entrepreneurship:

• Self-employment rate
• Kauffman Index of Entrepreneurial Activity
• Business startup rate
• Ratio of science and engineering bachelor degree holders to the total population
• Patents per science and engineering workforce
• Venture capital investment over Gross State Product
• Research and development expenditures
• Number of high-growth Inc. 500 firms

The indicators were chosen because they typically are used to measure entrepreneurship and innovation, two vital indicators for every state's economic health. But after employing several analyses, the researchers found that most of the eight indicators were only modestly correlated, if at all.

To prove the point, the authors invite readers to create their own state rankings. Demonstrating that rankings can be developed in ways that favor any given state, the Kauffman researchers created a simulation analysis with randomly generated weights, which revealed that, among 1,000 different scenarios, five states were eligible to be No. 1, 16 were eligible for the Top Five and 22 could be ranked in the Top 10.

The subjectivity and limitations of state economic rankings led to four observations:

• Policymakers should not rely on a single indicator to gauge economic conditions.
• Aggregating indicators does not provide solutions because indicators are highly variable.
• Policymakers should not focus on improving their states' rankings because the rankings lack meaning.
• Rather, they should employ a scorecard approach, which does not create a normative, quantified measure, but descriptively assesses various conditions of each state.
State economic rankings, though popular with think tanks, policymakers, media and even some academics, cannot be taken at face value. Such rankings typically are subjective and provide little meaningful information, despite their pervasiveness.

For example, the Small Business and Entrepreneurship Council's Small Business Survival Index – one of more than a dozen state rankings touted across the country – is based on factors such as taxes, health care costs, minimum wage level and the presence of right-to-work laws. However, this measures government costs without factoring in benefits generated by government, such as infrastructure and small business development centers.

Kauffman Foundation research has been used as the basis for ranking states, most notably on the rate of entrepreneurial activity, formalized in the Kauffman Index of Entrepreneurial Activity. This is an index based on data from the U.S. Census Bureau and Bureau of Labor Statistics, and it has never intended to be used to claim that one state or another is "better" for entrepreneurs.

Forthcoming Kauffman research shows that many rankings fail even to correlate with company owners' business climate perceptions. Further, the proliferation of state rankings can lead policymakers and economic development consultants to misuse them, either celebrating a conveniently positive ranking or initiating efforts to address a poor ranking, when neither action is based on valid economic indicators.

The authors conducted a series of exercises to demonstrate how such rankings can be manipulated. The exercises were based on eight state-level indicators related to innovation and entrepreneurship:

• Self-employment rate
• Kauffman Index of Entrepreneurial Activity
• Business startup rate
• Ratio of science and engineering bachelor degree holders to the total population
• Patents per science and engineering workforce
• Venture capital investment over Gross State Product
• Research and development expenditures
• Number of high-growth Inc. 500 firms

The indicators were chosen because they typically are used to measure entrepreneurship and innovation, two vital indicators for every state's economic health. But after employing several analyses, the researchers found that most of the eight indicators were only modestly correlated, if at all.

To prove the point, the authors invite readers to create their own state rankings. Demonstrating that rankings can be developed in ways that favor any given state, the Kauffman researchers created a simulation analysis with randomly generated weights, which revealed that, among 1,000 different scenarios, five states were eligible to be No. 1, 16 were eligible for the Top Five and 22 could be ranked in the Top 10.

The subjectivity and limitations of state economic rankings led to four observations:

• Policymakers should not rely on a single indicator to gauge economic conditions.
• Aggregating indicators does not provide solutions because indicators are highly variable.
• Policymakers should not focus on improving their states' rankings because the rankings lack meaning.
• Rather, they should employ a scorecard approach, which does not create a normative, quantified measure, but descriptively assesses various conditions of each state.

More info:

Published by: The Ewing Marion Kauffman Foundation on May 15, 2014
Copyright:Traditional Copyright: All rights reserved

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

09/29/2014

pdf

text

original

 
0
 
HOW CAN I CREATE MY FAVORITE STATE RANKING The Hidden Pitfalls of Statistical Indexes Yasuyuki Motoyama, Ph.D. and Jared Konczal
Ewing Marion Kauffman Foundation
 
September 2013
 
1
HOW CAN I CREATE MY FAVORITE STATE RANKING The Hidden Pitfalls of Statistical Indexes Yasuyuki Motoyama, Ph.D. and Jared Konczal
Ewing Marion Kauffman Foundation
 Reprinted from the
Journal of Applied Research in Economic Development
© 2013 by the Ewing Marion Kauffman Foundation. All rights reserved.
 
2
HOW CAN I CREATE MY FAVORITE STATE RANKING The Hidden Pitfalls of Statistical Indexes State economic rankings are both pervasive and popular. The long list of well-known rankings includes the State Competitiveness Report by the Beacon Hill Institute, the State New Economy Index by the Information Technology and Innovation Foundation,[1] the Cost-of-Doing-Business Index by the Milken Institute, the Economic Freedom Index by the Pacific Research Institute, and the State Business Tax Climate Index by the Tax Foundation, among many others. These rankings
 –
and even many that are less familiar 
 –
receive substantial media attention on the scale of thousands of viewers each day. As a result, media outlets have started to create their own rankings:
Forbes’ Best States for Business and CNBC’s America’s Top States for
Business. We know that there are more than one dozen out there, and the list seems to grow every year. [1] [Disclosure: the Kauffman Foundation has given grant funding to support this report in the past. Also, Kauffman research has been used as the basis for ranking states, most notably on the rate of entrepreneurial activity, formalized in the Kauffman Index of Entrepreneurial Activity. This is an index-based data from the U.S. Census Bureau and Bureau of Labor Statistics, and has never been a normative attempt to claim that one
state or another is “better” for entrepreneurs
.]
 
So business climate indexes have become a cottage industry within the economic development profession. Reacting to, and explaining why and how their state-community is ranked by business climate indexes has accordingly become a cottage industry among state and sub-state economic developers. The combination of
publishing indexes and states and community’s reacting to them has not been a pretty
picture
 –
in fact, it can be quite challenging and sometimes demeaning. Frankly, a serious measure of cynicism, opportunism, confusion, and manipulation has also entered this dismal business climate index picture. A large part of the business
climate index “problem” is that, at heart, business climate indexes are a statistical and
methodological concoction drawn from a database that is unavailable to the economic developer, the media, or the governance of your agency-community. We, as state and sub-state economic developers, are pretty much at the mercy of the entity and researcher compiling and presenting the index. Understanding statistical findings derived from poorly explained methodologies is usually not the forte of a state and sub-state economic developer. Hence this article. Statistics and methodologies are a magical land that badly needs some explanation.

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->