Table of Contents
About the Agricultural Resource Management Survey (ARMS)
The Agricultural Resource Management Survey (ARMS) is sponsored jointly by USDA's Economic Research Service (ERS) and National Agricultural Statistics Service (NASS). ARMS was first conducted in 1996. It combined USDA's previous cropping practices, chemical use, and farm costs and returns surveys, which were conducted separately from 1975 to 1995. ARMS is a multiphase series of interviews with farm operators about their cropping practices, farm businesses, and households.
Phase I–Screening Survey
Farmers and ranchers selected for the survey are contacted to verify that they still qualify as a farm and that they produce the specific commodities targeted by Phase II that year.
Phase II–Production Practices and Costs Report
Phase II questionnaires are only directed to operations producing the survey year’s target crop(s). The target crop is rotated on a multiyear cycle. Data are periodically collected for each major commodity, e.g. corn producers were surveyed in 2001, 2005, and 2010. Common information collected for all surveyed commodities includes (but is not limited to):
Field characteristics: field acreage, ownership, seed use and costs, yield, organic practices, conservation practices, crop rotations, and crop insurance.
Nutrient or fertilizer applications: applications of synthetic and organic nutrients, including application costs, amounts, methods, and timing; as well as manure, compost, and soil testing practices.
Bio-control or pesticide applications, and other pest management practices: applications of biological and chemical pesticides, including costs, amounts, methods, and timing; as well as various pest management practices such as scouting, assessment of pest problems, use of tillage and crop rotation, maintenance of equipment.
Field operations: use of machinery and labor in the field. Tillage, technical services, and the use of precision technologies are included in this section. Associated costs for fuel and labor are documented.
Irrigation: irrigation practices such as amount of water used, system type, and related expenses.
Phase III–Farm Business and Farm Household Information
Beginning in 2012, the Phase III uses only two questionnaire versions: a general Costs and Returns Report and an expanded Costs and Returns Report for target crop or livestock producers, which collects detailed commodity-specific information. Prior to 2012, a Core (short) version of the questionnaire was used to simplify the general Costs and Returns Report. Information collected on all versions includes (but is not limited to):
Land in farm/ranch: acres of land owned or rented, rents paid or received.
Acreage and production: acres harvested and total production.
Livestock: quantity sold or removed and ending quantity.
Commodity marketing and income: quantities and prices associated with marketing contracts, production contracts, cash or open market sales.
Other farm-related income: Federal farm programs, custom work, energy lease and royalty payments.
Operating and capital expenditures: livestock purchases, feed, seed, fertilizer, chemicals, fuel, labor, taxes, etc.
Farm assets: dwellings, farm buildings, land, inventory, equipment.
Farm debt: outstanding loans and their terms.
Farm management and use of time: farm legal organization, farm business ownership, number of operators, time allocation of operator and spouse.
Farm household: age and education of the operator and spouse, nonfarm income, assets, and debt.
ARMS is a multiphase, multiframe, stratified, probability-weighted sampling design. What do these four characteristics of the survey design mean?
ARMS is a series of interviews with farm operators about their management practices, farm business structure and finances, and household characteristics. It is conducted annually in three phases:
Phase I: conducted during the summer of the reference year.
Farmers selected for the survey sample are contacted to verify their operating status and whether they produced the commodities being surveyed that year. Phase I improves survey efficiency.
Phase II: conducted in the fall and winter of the reference year.
Randomly selected operating farms from Phase I are interviewed regarding their production practices and chemical use. Phase II data are collected at the individual field or production unit level and include physical and economic data on production inputs, management practices, and commodity costs of production.
Phase III: conducted in the early spring of the year following the reference year.
A nationally representative sample of farmers, randomly selected from Phase I, is interviewed to collect information on the characteristics and finances of all farm businesses and farm households during the reference year. Phase III data are collected at the whole-farm level and, for the livestock-specific version, at the livestock-enterprise level. As a part of the larger Phase III sample, operators that complete a Phase II questionnaire are also asked to complete a Phase III questionnaire. This provides information on their costs and returns, including data needed to estimate the costs associated with their production practices. The costs of production estimates include the enterprise-share of farm business expense items, such as land taxes, insurance, fuel expenses, etc., collected only in the Phase III interviews.
NASS uses two sampling frames to sample a total of about 30,000 farm and ranch operations for ARMS each year. The majority of farms—around 94 percent—come from the List Frame.
- List Frame—The primary sample is derived from the NASS List Frame. NASS maintains a list of farm operations that exhibit certain characteristics. The list is constructed and maintained from many different sources, including the Census of Agriculture and other NASS surveys. Because some information is already known about these farms, the list can be sorted according to farm types and size classes. For more information on the List Frame, see List Frame Samples on the NASS website.
- Area Frame—The NASS Area Frame supplements the List Frame. The farm population continually evolves as some farms change ownership or exit and new farms emerge. At any point in time, the List Frame invariably misses some farms, which the Area Frame helps capture. The Area Frame consists of farms captured through a random selection of agricultural land segments. NASS annually conducts a spring survey selected from the Area Frame to estimate crop acreage and land use. For more information on the Area Frame, see Area Frame Samples on the NASS website.
Strata are divisions within the sample frames that group farms by region (or by State in the case of 15 heavily sampled States), farm sales category, and commodity specialization. Farms in different strata are sampled with a different probability of selection to ensure that each sample includes a sufficient number of farms in different regions, sizes, and commodities. Within a stratum, the weight (expansion factor) is based on the probability of each sampled unit’s selection.
Because the ARMS sample is not a simple random sample, each observation has a different weight, or expansion factor, to reflect its probability of selection and, therefore, what part of the sampled universe it represents. Appropriate sample weights (expansion factors) are provided to prepare population estimates from the survey results. Population estimates are constructed by weighting each sample with the appropriate expansion factor. A jackknife re-sampling process is used with 30 additional weights from NASS for each sample to estimate the variances for each data item.
Furthermore, data from the Phase II of ARMS is divided into three data files: 1) fertilizers, 2) pesticides, and 3) all other data designated as the main file (e.g., field characteristics, management practices, and production input data other than fertilizers and pesticides). Sample weights associated with each of the three data files depend on the number of usable responses for the respective parts of the Phase II questionnaire. The usability of these tables for the construction of chemical or fertilizer use estimates is determined independently from the completion of the remainder of the questionnaire. Typically, slightly different response rates exist for these three parts of the questionnaire, and hence, weights differ between the main file and the two subfiles (pesticide and fertilizer). Cross-tabbing of variables across the three data files can give different population estimates for the same variable. In general, such population estimate differences across tables are relatively small. For more detail, see Farm Production Expenditures Methodology and Quality Measures PDF icon (16x16) .
Questionnaires and Manuals
Questionnaires are printed forms or computer programs used to ask specific questions and record the responses given by the sampled respondents. Due to the complex design of the ARMS survey, multiple questionnaires are used each year. Phase I questionnaires are used to collect current information for accurate sampling. All complete Phase II respondents are asked to complete a Phase III follow-on report. Every effort is made to ensure that both the Phase II and Phase III questionnaires are completed for these operations in both samples. Data from both phases provide the link between agricultural resource use and farm financial conditions, one of the cornerstones of the ARMS design. ERS and NASS jointly develop the questionnaires.
Manuals are produced each year to instruct enumerators and editors, and provide survey documentation. Both questionnaires and manuals can be found through visiting the ARMS questionnaires and manuals page.
Time: Each year's survey asks for information from the prior year. For Phase II, the reference year corresponds to the crop's production cycle (from one harvest season to the next); for Phase III, the reference year is the calendar year (Jan. 1 to Dec. 31). ARMS has been administered every year since 1996, when it replaced the Farm Costs and Returns Survey.
Geography: ARMS collects information from farms in the 48 contiguous States. Some agriculturally important States are oversampled to provide enough sample coverage to allow representative selected State-level estimates. The larger samples were started in 2003 and data were first available in 2004. States that are oversampled are Arkansas, California, Florida, Georgia, Illinois, Indiana, Iowa, Kansas, Minnesota, Missouri, Nebraska, North Carolina, Texas, Washington, and Wisconsin. Other States are provided increased commodity specific samples to ensure crop or livestock coverage. See States surveyed by commodity and year below.
Farms: ARMS samples and collects information from farms of all sizes, family and nonfamily farms, and corporate and unincorporated farms. It does not collect information from prison farms or research farms. To receive the survey and have responses incorporated into the ARMS data, a place must qualify as a farm, which means a place from which $1,000 or more of agricultural products were produced and sold or normally would have been sold, during the year.
Households: ARMS only collects household information for the households of principal operators of family farms. The principal operator is the person who runs the farm, making the day-to-day management decisions. A family farm is where the principal operator and people related to the operator by blood, marriage, or adoption, have more than a 50-percent ownership interest in the farm.
Commodities: While farms growing all types of commodities are sampled every year, each year ARMS oversamples farms in one or more commodity specialization in order to produce Costs and Returns estimates. For sampled commodity crop producers, both a Phase II Production Practices and Costs Report and Phase III Costs and Returns Report are requested. For livestock commodities, only the farm-level Phase III questionnaire is necessary. The table below indicates which commodities were oversampled in each year since the inception of ARMS.
ARMS coverage by commodity and year
Commodity 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013
Corn checkmark_green.gif checkmark_yellow.gif checkmark_yellow.gif checkmark_yellow.gif checkmark_yellow.gif checkmark_green.gif checkmark_green.gif checkmark_green.gif
Soybeans checkmark_yellow.gif checkmark_green.gif checkmark_yellow.gif checkmark_yellow.gif checkmark_yellow.gif checkmark_green.gif checkmark_green.gif checkmark_green.gif
Cotton checkmark_yellow.gif checkmark_green.gif checkmark_yellow.gif checkmark_yellow.gif checkmark_yellow.gif checkmark_green.gif checkmark_green.gif
Winter Wheat checkmark_yellow.gif checkmark_yellow.gif checkmark_green.gif checkmark_yellow.gif checkmark_yellow.gif checkmark_green.gif checkmark_green.gif
Spring Wheat checkmark_yellow.gif checkmark_yellow.gif checkmark_green.gif checkmark_yellow.gif checkmark_green.gif checkmark_green.gif
Durum Wheat checkmark_yellow.gif checkmark_yellow.gif checkmark_green.gif checkmark_yellow.gif checkmark_green.gif checkmark_green.gif
Potatoes checkmark_yellow.gif checkmark_yellow.gif checkmark_yellow.gif checkmark_yellow.gif
Rice checkmark_green.gif checkmark_green.gif checkmark_green.gif
Sorghum (milo) checkmark_yellow.gif checkmark_green.gif checkmark_green.gif
Tobacco checkmark_green.gif checkmark_red.gif
Peanuts checkmark_yellow.gif checkmark_green.gif checkmark_green.gif
Barley checkmark_green.gif checkmark_green.gif
Cow-calf checkmark_red.gif checkmark_red.gif
Hogs checkmark_red.gif checkmark_red.gif checkmark_red.gif
Dairy checkmark_red.gif checkmark_red.gif checkmark_red.gif
Broilers checkmark_red.gif checkmark_red.gif
checkmark_yellow.gif = Phase II field-level Production Practices Report only.
checkmark_green.gif = Both Phase II field-level Production Practices Report and Phase III whole-farm Costs of Production survey.
checkmark_red.gif = Phase III whole-farm Costs of Production survey only.
Commodities by State: The States included in the commodity specific survey each year vary, depending on the crops surveyed, to help minimize respondent burden. The sampling used in ARMS Phase II was not intended to support State estimates, but sufficient data are obtained in some States to report these estimates. However, the ability to partition data for individual States is very limited. The table below indicates which commodities were oversampled in each State by year since the inception of ARMS.
States surveyed by commodity and year
2007 CA MI NY NC OR PA WA
1996 IL IN IA KS KY MI MN MO NE NC OH PA SC SD TX WI
1997 IL IN IA MI MN MO NE OH SD WI
1998 CO IL IN IA KS KY MI MN MO NE NC OH PA SD TX WI
1999 CO IL IN IA KS KY MI MN MO NE NC OH SD TX WI
2000 CO IL IN IA KS KY MI MN MO NE NY NC ND OH PA SD TX WI
2001 CO GA IL IN IA KS KY MI MN MO NE NY NC ND OH PA SD TX WI
2005 CO GA IL IN IA KS KY MI MN MO NE NY NC ND OH PA SD TX WI
2010 CO GA IL IN IA KS KY MI MN MO NE NY NC ND OH PA SD TX WI
1996 AR IL IN IA LA MN MS MO NE OH TN WI
1997 AR DE IL IN IA KS KY LA MI MN MS MO NE NC OH PA SD TN WI
1998 AR IL IN IA KS KY LA MI MN MS MO NE NC OH SD TN
1999 AR IL IN IA KS KY LA MI MN MS MO NE NC OH PA SD TN
2000 AR IL IN IA KS KY LA MI MN MS MO NE NC ND OH SD TN WI
2002 AR IL IN IA KS KY LA MD MI MN MS MO NE NC ND OH SD TN VA WI
2006 AR IL IN IA KS KY LA MI MN MS MO NE NC ND OH SD TN VA WI
2012 AR IL IN IA KS KY LA MI MN MS MO NE NC ND OH SD TN VA WI
1996 AZ AR CA GA LA MS TN TX
1997 AL AZ AR CA GA LA MS MO NC SC TN TX
1998 AL AZ AR CA GA LA MS NC TN TX
1999 AL AZ AR CA GA LA MS NC TN TX
2000 AL AZ AR CA GA LA MS MO NC TN TX
2003 AL AZ AR CA GA LA MS MO NC SC TN TX
2007 AL AR CA GA LA MS MO NC SC TN TX
1998 CA MT ND SD
2004 MT ND
2009 ID MT ND SD
Other Spring Wheat
1996 MN MT ND
1997 MN MT ND SD
1998 ID MN MT ND OR SD WA
2000 MN MT ND SD
2004 ID MN MT ND OR SD WA
2009 CO ID MN MT ND OR SD WA
1996 CO DE ID KS MT NE OK OR SD TX WA
1997 CO ID IL KS MO MT NE OH OK OR PA SD TX WA
1998 CA CO GA ID IL KS LA MN MS MO MT NE NC OH OK OR SD TX WA
2000 AR CO ID IL KS KY MO MT NE NC OH OK OR SD TX WA
2004 CO ID IL KS MI MO MT NE OH OK OR SD TX WA
2009 CO ID IL KS MI MN MO MT NE ND OH OK OR SD TX WA
1999 AL GA NC TX
2004 AL FL GA NC TX
2013 AL FL GA SC NC TX
1996 ID ME WA
1997 ID ME MN ND OR WA WI
1998 PA WI
2000 AR CA LA MS TX
2006 AR CA LA MS MO TX
2013 AR CA LA MS MO TX
2000 CA CO ID MI MN MT NE ND OR WA WY
1999 KS ND SD
2005 IL IA KS MI MN NE NY ND PA SD TX WI
2003 CA ID MN MT ND PA SD UT WA WI WY
2011 AZ CA CO ID MN MT ND OR PA VA WA WI WY
2003 CO KS MO NE OK SD TX
2011 CO KS NE OK SD TX
1996 GA NC SC VA
2008 GA KY NC SC TN VA
1996 CA CO FL ID IA IL KS KY MO MT NE NM ND OK OR SD TN TX WY
2008 AL AR CA CO FL GA IA KS KY MO MS MT NE NM ND OK OR SD TN TX VA WY
1998 AL AR CO GA IL IN IA KS KY MI MN MO NE NC OH OK SC SD TN UT VA WI
2004 AR CO GA IL IN IA KS KY MI MN MO NE NC OH OK PA SD VA WI
2009 AR CO GA IL IN IA KS KY MI MN MO NE NC OH OK PA SD VA WI
2000 AZ CA FL GA ID IL IN IA KY MI MN MO NM NY OH PA TN TX VT VA WA WI
2005 AZ CA FL GA ID IL IN IA KY MI MN MO NM NY OH PA TN TX VT VA WA WI
2010 AZ CA CO FL GA ID IL IN IA KS KY ME MI MN MO NM NY OH OR PA TN TX VT VA WA WI
2006 AL AR CA DE GA KY LA MD MO MS NC OK PA SC TN TX VA
2011 AL AR CA DE GA KY LA MD MO MS NC OK PA SC TN TX VA
ARMS data are collected by self-administered mailed questionnaires and personal interviews conducted by trained enumerators using several types of questionnaires. Mail responses can be returned on the printed form or through electronic data reporting (EDR). Data collection starts in June with Phase I, continues during the fall with Phase II (production practices and cost data), and finishes in late winter (after the close of the year) with Phase III (whole-farm income and costs data). Phase II data are collected at the individual field and production unit level, while Phase III is collected for the whole farm business. Survey instruments (available for download) show the range of questions asked for Phase II and Phase III on income, expenses, assets, debt, and specific crop/livestock management practices. Interviewers' manuals (also available for download) outline detailed enumeration procedures for each phase of the survey, providing specific instruction on how to conduct an interview and specific methods and examples on recording complicated data. Questionnaires include specific notes with each question to guide respondents.
Data Processing and Analysis
Data collection and preliminary editing/analysis and processing are managed by NASS Headquarters in Washington, D.C. and by 12 NASS regional offices throughout the contiguous U.S. About 200-300 NASS personnel as well as ERS personnel are involved in preliminary editing/analysis and processing prior to delivery of a raw survey file to ERS.
Initial processing includes manual review and possible imputation by enumerators, supervisory enumerators, or regional statisticians for missing items or items that are not allowed to be initially coded as a nonresponse (-1). NASS then uses a set of automated processing tools, including FEITH, which provides scanned images of all questionnaires; PRISM2, an interactive data review system; and IDAS (Interactive Data Analysis System) for data review at the individual (respondent) and aggregated (strata, State, etc.) data levels.
Additional post-survey processing includes computer imputation to replace nonresponse codes for a number of items, survey weight calibration, outlier adjustment, and final summarization. ERS conditions the final raw survey data file with additional data editing, analysis, item imputation, and variable creation for use in research.
Survey enumerators, supervisory enumerators, and NASS regional office personnel provide a preliminary edit of completed ARMS questionnaires before entry into the NASS computer system. The questionnaires are then scanned into FEITH and data are manually keyed before being processed with a computerized edit program used to identify potential data errors. NASS staff in Washington and in regional offices, assisted by ERS staff, then check for errors and consistency using the PRISM2 editing program. After further analysis, NASS passes a completed raw survey file to ERS. ERS then creates a final dataset that includes some retained NASS variables, all keyed data, and several hundred additional calculated variables covering farm operation and farm operator household characteristics. ERS performs a final round of review, analysis, and updates of the complete ARMS research file prior to its release.
NASS Computer Edit/Analysis
The NASS computer edit identifies potential errors involving known physical relationships, obvious coding errors, and basic economic relationships between interrelated questionnaire items. Specifications for the computer edit are reviewed and updated annually by ERS and NASS. Errors/potential errors are divided into two categories—critical errors (the item value must be corrected or a comment explaining the value must be recorded), and warnings (the item value should be reviewed and/or updated).
NASS uses a combination of software tools including PRISM, FEITH, and IDAS to examine and evaluate ARMS data during the editing phase. IDAS is the principal tool for examination of individual report data. IDAS provides tabular and graphical displays of survey-level report items and additional analysis variables by State, region, and at the U.S. level. The objective is to facilitate the identification of remaining data errors. Custom Hyperion IR queries are also used if specific issues arise during analysis. Both ERS and NASS personnel participate in the editing and analysis of ARMS data with IDAS.
Survey responses are divided into two categories, those for which a value must be provided (either by the respondent or manually imputed by a NASS statistician) and those that can be initially coded as a nonresponse (the respondent refused to provide or did not know the answer). Items for which a value must be initially provided constitute a relatively small share (typically less than 30 percent) of all items. Statisticians are instructed to manually impute values for these items using one or more methods/sources including relationships in the questionnaire, similar reports, other State surveys, other outside sources and publications, and data from the ERS website.
Most of the items on the ARMS Phase III survey can be initially coded as a nonresponse (the respondent did not know or refused to provide an answer to an item). In 2005, for example, about 75 percent of all items fell in this category. At the request of ERS, NASS uses a computer algorithm to impute data for a small subset of these items (144 in 2011)—typically, about 10 percent of all items that can be coded as nonresponses. The items selected by ERS are based on its mission requirements for time sensitive development of farm operation/farm operator household financial and structural characteristics and by NASS’s need to complete analysis and the Farm Production Expenditures Summary publication. The remaining missing items are coded with a value of -1 in the file delivered to ERS.
The NASS imputation algorithm computes unweighted mean values for donors (farms reporting a positive value for an item) in a specific set of farm groupings based on location, farm size, and farm type. Donors with “extreme values” are excluded from calculations. In assigning an imputed value, the algorithm uses the first grouping (starting with group 1) that contains at least 10 donors. In 2012, 80 percent of imputed values were obtained from group 1. Means are computed for 12 imputation groups:
- Region, Sales Class, Farm Type
- Region, Pooled Sales Class, Farm Type
- U.S., Sales Class, Farm Type
- U.S., Pooled Sales Class, Farm Type
- Region, Sales Class, Pooled Farm Type
- Region, Pooled Sales Class, Pooled Farm Type
- U.S., Sales Class, Pooled Farm Type
- U.S., Pooled Sales Class, Pooled Farm Type
- Region, Farm Type
- Region, Sales Class
- Region Average
- U.S. Average
Farm depreciation expense, landlord taxes, and other assets are examples of individual items most frequently imputed. The remaining items for which data are most commonly imputed can be grouped into three categories: farm labor, other farm-related income, and farm assets.
After editing and imputation, survey sample weights are calibrated so that weighted survey totals for selected items match official USDA estimates for production or acreage where possible. A decision rule is employed to exclude calibration targets for survey items whose weighted totals fall below or above predetermined thresholds in relation to official estimates. The calibration items in the last few years have numbered between 30-32 targets as follows:
- Number of Farms by 8 economic classes
- Number of Farms by nonestimate States
- Number of Farms - Total
- Acres - Corn for grain
- Acres - Wheat for grain acres
- Acres - Soybean acres
- Acres - Cotton acres
- Acres - Vegetable acres
- Acres - Fruit acres
- Acres - Hay acres
- Acres - Peanut acres
- Acres - Rice acres
- Acres - Sugarcane/sugarbeet acres
- Acres - Tobacco acres
- Acres - Nursery/floriculture (2 targets)
- Acres - Costs of production crop target added that year for each crop not listed above
- Cattle on feed
- Hogs and pigs
Following calibration, outliers are identified and reviewed by the official USDA-NASS National Outlier Review Board. There are typically five or fewer national outliers each year. An outlier is defined as a sampled farm where weighted (expanded) data for total expenses accounts for 0.5 percent of U.S. total expenses and/or 2.5 percent of regional total expenses. The review board usually adjusts outlier sample weights downward. The general rule for treating outliers is to reweight to the median weight of the matching economic class by farm type. Following the National Outlier Review Board, outlier weights are frozen at board levels and the calibrated summary is rerun. Following recalibration, survey weights are rechecked for the possible introduction of new outliers. An iterative procedure is used to adjust the weights of the new outliers, and recalibrate the weights until outliers are mitigated.
ERS Dataset Conditioning
Following receipt of the raw survey file from NASS, ERS creates a preliminary version of the final ERS research file. The majority of NASS calculated variables are removed mainly because they cannot be updated if changes are made. Conditioning programs calculate several hundred chronologically and methodologically consistent financial and operational variables and add them to the raw survey data file. The additional research variables are typically calculated from a combination of raw survey items. ERS adds geographic identifiers (such as ERS farm resource regions), demographic information (for example, categorical variables based on the U.S. 2010 Population Census), and other county, regional, and national economic information. ARMS III data files are available for 1996 to present.
In creating the final research file, ERS imputes data for about 40 items that are necessary to meet time-sensitive ERS mission requirements for the publication of farm operation and farm operator household data. This includes farm operator debt and items pertaining to the farm operator and the farm operator’s household. Collectively, ERS and NASS impute data for about 15 percent of all items that could be coded as nonresponses. The remaining items are retained with a “-1” code in the final ERS research file.
In general, ERS employs the same basic methodology as NASS, using mean values for responding farms to replace refusal codes. ERS imputation schemes can be procedurally complex incorporating other survey items and varying in some cases by questionnaire version and the level of item aggregation. ERS imputes values for a number of items related to the farm operator and operator’s household, including:
- Off-farm income items—wages and salaries, net income from other farms, net income from other business, net income from renting farm land, interest income, dividend income, capital gains/loss, private retirement income, public retirement income, and other
- Share of farm income retained by operator household
- Farm operation debt
- Operator household size
- Household expenses, nonfarm assets, and nonfarm debt
- Total value farm sales, net operating income, and total off-farm income previous year.
ERS uses a larger set of classification variables (in addition to farm size, type, and location) in imputing household items, including the number of operators, the operators’ age class, education level, marital status, and retirement status, as well as the farms’ legal organization. Differences in questionnaire versions are also incorporated into imputation methods for the farm operator’s household.
ERS Data Review
Following creation of the preliminary version of the research data file, ERS performs a final review of the data. Key financial and operational values and relationships are audited for internal (within record) and external (across records and across time) consistency. SAS programs are used to identify errors in the computer code used to create the final research file and remaining problems in the data that were not identified during the NASS edit/analysis. Some records are updated by the conditioning programs while others are researched again at NASS and updates applied if errors are found. In evaluating potential data edits ERS also uses the auxiliary comment file provided by NASS. The comment file includes explanations at the survey record (farm) level by ERS and NASS record analysts about enumerator comments or “unusual data values” remaining in the survey file.
ERS and NASS provide "train the trainer" workshops prior to data collection for each survey. Regional and State statisticians then train enumerators through a series of dispersed workshops. ERS and NASS develop and provide training materials to the State survey statisticians. After questionnaires are completed by the enumerators, each questionnaire is reviewed by supervisory enumerators for completeness, inconsistent responses, or errors, and then transferred to a NASS State or Regional office where each questionnaire is reviewed before it is keyed into an electronic format. While the survey database is assembled, a computer edit is used to identify potential recording errors or inconsistencies in data relationships (for example, interest expense matched with farm debt). In the process of conditioning the survey database, uncharacteristic responses are investigated and data are verified or corrected. Select cells can be marked for later computer imputation. The Interviewer’s manual and the Survey Administration manual are updated each year and provide specific details to provide uniform treatment of data collection and data processing procedures.
Changes in Methodology
Time period Change
2013 The survey was redesigned to reduce expenditures while maintaining sample size. The Core version of ARMS Phase III was eliminated, and questionnaire length in other Phase III versions was reduced. Remaining versions shifted to mail mode, while retaining enumerator assistance on request.
2012 The ARMS questionnaire was integrated with the Census of Agriculture; ARMS questionnaires included all census questions, and primary ARMS questions.
2010-present New top-end value codes were introduced for other farm assets and off-farm income, assets, and debt, replacing the top code of 1.5 million and above that had been in place since 1996. The new top code is 10 million and above, and 4 additional codes were introduced to cover values between 1.5 million and 9.99 million.
2007 The ARMS questionnaire was integrated with the Census of Agriculture; ARMS questionnaires included all census questions, and primary ARMS questions.
2003-2012 A new, shorter, core version of the survey was introduced, as a mail-out survey with enumerator follow-up upon request. The added sample size allowed for State-level estimates to be generated for 16 major agricultural States. The number of useable responses rose to 20,000 or more after core introduction, compared to 10,000 before.
2002 The ARMS questionnaire was integrated with the Census of Agriculture; ARMS questionnaires included most census questions, and primary ARMS questions.
ERS and NASS follow strict statistical procedures when designing, collecting, and summarizing ARMS data. The observations from any given sample of the population must be weighted properly to ensure that sample estimates of totals and averages are representative of the total farm population. Through a process known as calibration survey weights are adjusted so that sample estimates of population totals hit known targets such as the total acres of corn.
Estimating variances of sample estimates gives an indication of how an estimate may vary if a different ARMS sample were drawn. An estimate’s variance gives a sense of how close a sample estimate is likely to be to the true population value. Given the complex design of ARMS, a specific method for estimating variances, known as the Delete-a-Group Jackknife, is recommended for calculating the variance of a sample estimate.
The following documents include information on the calibration of sample weights and variance estimation.
Variance Estimation with USDA’s Farm Costs and Returns Surveys and Agricultural Resource Management Study Surveys. This paper is an overview of survey estimators, sample design, hypothesis testing, disclosure rules, and reliability measures for ARMS followed by statistical program documentation. Sums, ratios, means, multiple regression, binomial logit analysis, and order statistics are covered. Contact Robert Dubman for a copy of this paper.
As with most surveys, some sampled farm operators do not respond to particular questions (item nonresponse) or to the entire survey (unit nonresponse). For some questions, item nonresponse is addressed by imputation, which is a statistical procedure that uses information from other questions to fill in missing values. Unit nonresponse, in contrast, is at least partially addressed through adjustments to sample weights, including calibration, that help to account for the missing farms.
The following publications provide information such as patterns in nonresponse and potential consequences and solutions.
Several documents are available which explain in technical detail various issues related to the ARMS.
Farm Production Expenditures Methodology and Quality Measures, PDF icon (16x16) 2013.This document explains the methodology and quality measures for NASS estimates of farm production expenditures, covering topics such as sampling, data collection methods, nonresponse adjustments, calibration, and outliers.
An Economist’s Primer on Survey Samples, PDF icon (16x16) 2000. This document provides an accessible overview of how the implications of incorporation sample design into economic analysis of survey data.
This set of maps helps data users visualize and better understand the geographical scope and level of aggregation by which many ARMS data are summarized. For example, the ARMS web data tool presents farm financial estimates aggregated to Farm Resource Regions, which do not follow State boundaries. Maps below demonstrate the different ways data are often summarized.
U.S. Farm Resource Regions: Farm resource regions are defined using farm production regions, land resource regions, crop reporting districts, and farm characteristics. The regions are designated at the county level and therefore do not generally follow State boundaries. See Farm Resource Regions, AIB-760, August 2000.
Farm Resource Regions
Farm Production Regions: The older Farm Production Regions, in following State boundaries, group unlike areas together because a single State often encompasses different soils and typography. For example, the old Appalachian Region, comprised of Tennessee, Kentucky, North Carolina, West Virginia, and Virginia, contains the Appalachian Mountains, Piedmont, and Coastal Plain areas, all of which have quite different agriculture.
Farm Production Regions
Patterns of Agricultural Diversity: County clusters, based on types of commodities produced, have shown that a few commodities tend to dominate farm production in specific geographic areas that cut across State boundaries. The climate, soil, water, and topography in localized geographic areas tend to constrain the types of crops and livestock that will thrive there. See Diversity in U.S. Agriculture: A New Delineation by Farming Characteristics PDF icon (16x16) , AER-646, July 1991, for more information.
Cluster analysis of farm characteristics
USDA Land Resource Regions: In constructing the ERS production regions, analysts identified where areas with similar types of farms intersected with areas of similar physiographic, soil, and climatic traits, as reflected in USDA's Land Resource Regions.
USDA land resource regions
NASS Crop Reporting Districts: County reporting districts influenced the construction of the ERS farm resource regions by conforming intersecting areas to follow the boundaries of NASS Crop Reporting Districts (CRD), which are aggregates of counties. With more and more data available at the county level, geographic representations need no longer be constrained to follow State boundaries.
NASS crop reporting districts
NASS ARMS Regions: Five Farm Production Expenditure Regions