Topics Topics

Documentation

Table of Contents

About the Agricultural Resource Management Survey (ARMS)

The Agricultural Resource Management Survey (ARMS) is sponsored jointly by USDA's Economic Research Service (ERS) and National Agricultural Statistics Service (NASS). ARMS was first conducted in 1996. It combined USDA's previous cropping practices, chemical use, and farm costs and returns surveys, which were conducted separately from 1975 to 1995. ARMS is a multiphase series of interviews with farm operators about their cropping practices, farm businesses, and households.

Phase I–Screening Survey

Farmers and ranchers selected for the survey are contacted to verify that they still qualify as a farm and that they produce the specific commodities targeted by Phase II that year.

Phase II–Production Practices and Costs Report

Phase II questionnaires are only directed to operations producing the survey year’s target crop(s). The target crop is rotated on a multiyear cycle. Data are periodically collected for each major commodity, e.g. corn producers were surveyed in 2001, 2005, and 2010. Common information collected for all surveyed commodities includes (but is not limited to):

  • Field characteristics: field acreage, ownership, seed use and costs, yield, organic practices, conservation practices, crop rotations, and crop insurance.
  • Nutrient or fertilizer applications: applications of synthetic and organic nutrients, including application costs, amounts, methods, and timing; as well as manure, compost, and soil testing practices.
  • Bio-control or pesticide applications, and other pest management practices: applications of biological and chemical pesticides, including costs, amounts, methods, and timing; as well as various pest management practices such as scouting, assessment of pest problems, use of tillage and crop rotation, maintenance of equipment.
  • Field operations: use of machinery and labor in the field. Tillage, technical services, and the use of precision technologies are included in this section. Associated costs for fuel and labor are documented.
  • Irrigation: irrigation practices such as amount of water used, system type, and related expenses.

Phase III–Farm Business and Farm Household Information

Beginning in 2012, the Phase III uses only two questionnaire versions: a general Costs and Returns Report and an expanded Costs and Returns Report for target crop or livestock producers, which collects detailed commodity-specific information. Prior to 2012, a Core (short) version of the questionnaire was used to simplify the general Costs and Returns Report. Information collected on all versions includes (but is not limited to):

  • Land in farm/ranch: acres of land owned or rented, rents paid or received.
  • Acreage and production: acres harvested and total production.
  • Livestock: quantity sold or removed and ending quantity.
  • Commodity marketing and income: quantities and prices associated with marketing contracts, production contracts, cash or open market sales.
  • Other farm-related income: Federal farm programs, custom work, energy lease and royalty payments.
  • Operating and capital expenditures: livestock purchases, feed, seed, fertilizer, chemicals, fuel, labor, taxes, etc.
  • Farm assets: dwellings, farm buildings, land, inventory, equipment.
  • Farm debt: outstanding loans and their terms.
  • Farm management and use of time: farm legal organization, farm business ownership, number of operators, time allocation of operator and spouse.
  • Farm household: age and education of the operator and spouse, nonfarm income, assets, and debt.

Survey Design

ARMS is a multiphase, multiframe, stratified, probability-weighted sampling design. What do these four characteristics of the survey design mean?

Multiphase

ARMS is a series of interviews with farm operators about their management practices, farm business structure and finances, and household characteristics. It is conducted annually in three phases:

  • Phase I: conducted during the summer of the reference year.
    Farmers selected for the survey sample are contacted to verify their operating status and whether they produced the commodities being surveyed that year. Phase I improves survey efficiency.
  • Phase II: conducted in the fall and winter of the reference year.
    Randomly selected operating farms from Phase I are interviewed regarding their production practices and chemical use. Phase II data are collected at the individual field or production unit level and include physical and economic data on production inputs, management practices, and commodity costs of production.
  • Phase III: conducted in the early spring of the year following the reference year.
    A nationally representative sample of farmers, randomly selected from Phase I, is interviewed to collect information on the characteristics and finances of all farm businesses and farm households during the reference year. Phase III data are collected at the whole-farm level and, for the livestock-specific version, at the livestock-enterprise level. As a part of the larger Phase III sample, operators that complete a Phase II questionnaire are also asked to complete a Phase III questionnaire. This provides information on their costs and returns, including data needed to estimate the costs associated with their production practices. The costs of production estimates include the enterprise-share of farm business expense items, such as land taxes, insurance, fuel expenses, etc., collected only in the Phase III interviews.

ModularARMS.fw.png

Multiframe

NASS uses two sampling frames to sample a total of about 30,000 farm and ranch operations for ARMS each year. The majority of farms—around 94 percent—come from the List Frame.

  • List Frame—The primary sample is derived from the NASS List Frame. NASS maintains a list of farm operations that exhibit certain characteristics. The list is constructed and maintained from many different sources, including the Census of Agriculture and other NASS surveys. Because some information is already known about these farms, the list can be sorted according to farm types and size classes. For more information on the List Frame, see List Frame Samples on the NASS website.
  • Area Frame—The NASS Area Frame supplements the List Frame. The farm population continually evolves as some farms change ownership or exit and new farms emerge. At any point in time, the List Frame invariably misses some farms, which the Area Frame helps capture. The Area Frame consists of farms captured through a random selection of agricultural land segments. NASS annually conducts a spring survey selected from the Area Frame to estimate crop acreage and land use. For more information on the Area Frame, see Area Frame Samples on the NASS website.

Stratified

Strata are divisions within the sample frames that group farms by region (or by State in the case of 15 heavily sampled States), farm sales category, and commodity specialization. Farms in different strata are sampled with a different probability of selection to ensure that each sample includes a sufficient number of farms in different regions, sizes, and commodities. Within a stratum, the weight (expansion factor) is based on the probability of each sampled unit’s selection.

Probability-weighted

Because the ARMS sample is not a simple random sample, each observation has a different weight, or expansion factor, to reflect its probability of selection and, therefore, what part of the sampled universe it represents. Appropriate sample weights (expansion factors) are provided to prepare population estimates from the survey results. Population estimates are constructed by weighting each sample with the appropriate expansion factor. A jackknife re-sampling process is used with 30 additional weights from NASS for each sample to estimate the variances for each data item.

Furthermore, data from the Phase II of ARMS is divided into three data files: 1) fertilizers, 2) pesticides, and 3) all other data designated as the main file (e.g., field characteristics, management practices, and production input data other than fertilizers and pesticides). Sample weights associated with each of the three data files depend on the number of usable responses for the respective parts of the Phase II questionnaire. The usability of these tables for the construction of chemical or fertilizer use estimates is determined independently from the completion of the remainder of the questionnaire. Typically, slightly different response rates exist for these three parts of the questionnaire, and hence, weights differ between the main file and the two subfiles (pesticide and fertilizer). Cross-tabbing of variables across the three data files can give different population estimates for the same variable. In general, such population estimate differences across tables are relatively small. For more detail, see Farm Production Expenditures Methodology and Quality Measures PDF icon (16x16) .

Questionnaires and Manuals

Questionnaires are printed forms or computer programs used to ask specific questions and record the responses given by the sampled respondents. Due to the complex design of the ARMS survey, multiple questionnaires are used each year. Phase I questionnaires are used to collect current information for accurate sampling. All complete Phase II respondents are asked to complete a Phase III follow-on report. Every effort is made to ensure that both the Phase II and Phase III questionnaires are completed for these operations in both samples. Data from both phases provide the link between agricultural resource use and farm financial conditions, one of the cornerstones of the ARMS design. ERS and NASS jointly develop the questionnaires.

Manuals are produced each year to instruct enumerators and editors, and provide survey documentation. Both questionnaires and manuals can be found through visiting the ARMS questionnaires and manuals page.

Survey Scope

  • Time: Each year's survey asks for information from the prior year. For Phase II, the reference year corresponds to the crop's production cycle (from one harvest season to the next); for Phase III, the reference year is the calendar year (Jan. 1 to Dec. 31). ARMS has been administered every year since 1996, when it replaced the Farm Costs and Returns Survey.
  • Geography: ARMS collects information from farms in the 48 contiguous States. Some agriculturally important States are oversampled to provide enough sample coverage to allow representative selected State-level estimates. The larger samples were started in 2003 and data were first available in 2004. States that are oversampled are Arkansas, California, Florida, Georgia, Illinois, Indiana, Iowa, Kansas, Minnesota, Missouri, Nebraska, North Carolina, Texas, Washington, and Wisconsin. Other States are provided increased commodity specific samples to ensure crop or livestock coverage. See States surveyed by commodity and year below.
  • Farms: ARMS samples and collects information from farms of all sizes, family and nonfamily farms, and corporate and unincorporated farms. It does not collect information from prison farms or research farms. To receive the survey and have responses incorporated into the ARMS data, a place must qualify as a farm, which means a place from which $1,000 or more of agricultural products were produced and sold or normally would have been sold, during the year.
  • Households: ARMS only collects household information for the households of principal operators of family farms. The principal operator is the person who runs the farm, making the day-to-day management decisions. A family farm is where the principal operator and people related to the operator by blood, marriage, or adoption, have more than a 50-percent ownership interest in the farm.
  • Commodities: While farms growing all types of commodities are sampled every year, each year ARMS oversamples farms in one or more commodity specialization in order to produce Costs and Returns estimates. For sampled commodity crop producers, both a Phase II Production Practices and Costs Report and Phase III Costs and Returns Report are requested. For livestock commodities, only the farm-level Phase III questionnaire is necessary. The table below indicates which commodities were oversampled in each year since the inception of ARMS.

ARMS coverage by commodity and year

Commodity    1996    1997    1998    1999    2000    2001    2002    2003    2004    2005    2006    2007    2008    2009    2010    2011    2012    2013   
Apples                          checkmark_green.gif                
Corn    checkmark_green.gif     checkmark_yellow.gif     checkmark_yellow.gif     checkmark_yellow.gif     checkmark_yellow.gif     checkmark_green.gif           checkmark_green.gif             checkmark_green.gif          
Soybeans    checkmark_yellow.gif     checkmark_green.gif     checkmark_yellow.gif     checkmark_yellow.gif     checkmark_yellow.gif       checkmark_green.gif           checkmark_green.gif               checkmark_green.gif      
Cotton    checkmark_yellow.gif     checkmark_green.gif     checkmark_yellow.gif     checkmark_yellow.gif     checkmark_yellow.gif         checkmark_green.gif           checkmark_green.gif                
Winter Wheat    checkmark_yellow.gif     checkmark_yellow.gif     checkmark_green.gif     checkmark_yellow.gif     checkmark_yellow.gif           checkmark_green.gif             checkmark_green.gif            
Spring Wheat    checkmark_yellow.gif     checkmark_yellow.gif     checkmark_green.gif       checkmark_yellow.gif           checkmark_green.gif             checkmark_green.gif            
Durum Wheat    checkmark_yellow.gif     checkmark_yellow.gif     checkmark_green.gif       checkmark_yellow.gif           checkmark_green.gif             checkmark_green.gif            
Potatoes    checkmark_yellow.gif     checkmark_yellow.gif     checkmark_yellow.gif     checkmark_yellow.gif                                
Rice            checkmark_green.gif               checkmark_green.gif                 checkmark_green.gif    
Sorghum (milo)        checkmark_yellow.gif             checkmark_green.gif                   checkmark_green.gif        
Tobacco    checkmark_green.gif                           checkmark_red.gif              
Sugarbeets            checkmark_green.gif                              
Peanuts          checkmark_yellow.gif             checkmark_green.gif                     checkmark_green.gif    
Sunflowers          checkmark_yellow.gif                                
Oats                      checkmark_green.gif                    
Barley                  checkmark_green.gif                   checkmark_green.gif        
Cow-calf    checkmark_red.gif                           checkmark_red.gif              
Hogs        checkmark_red.gif               checkmark_red.gif             checkmark_red.gif            
Dairy            checkmark_red.gif             checkmark_red.gif             checkmark_red.gif          
Broilers                        checkmark_red.gif             checkmark_red.gif        

checkmark_yellow.gif = Phase II field-level Production Practices Report only.
checkmark_green.gif = Both Phase II field-level Production Practices Report and Phase III whole-farm Costs of Production survey.
checkmark_red.gif = Phase III whole-farm Costs of Production survey only.

  • Commodities by State: The States included in the commodity specific survey each year vary, depending on the crops surveyed, to help minimize respondent burden. The sampling used in ARMS Phase II was not intended to support State estimates, but sufficient data are obtained in some States to report these estimates. However, the ability to partition data for individual States is very limited. The table below indicates which commodities were oversampled in each State by year since the inception of ARMS.

States surveyed by commodity and year

Apples   
2007    CA    MI    NY    NC    OR    PA    WA                                         
Corn   
1996        IL    IN    IA    KS    KY    MI    MN    MO    NE      NC      OH    PA    SC    SD    TX    WI               
1997        IL    IN    IA        MI    MN    MO    NE          OH        SD      WI               
1998    CO      IL    IN    IA    KS    KY    MI    MN    MO    NE      NC      OH    PA      SD    TX    WI               
1999    CO      IL    IN    IA    KS    KY    MI    MN    MO    NE      NC      OH        SD    TX    WI               
2000    CO      IL    IN    IA    KS    KY    MI    MN    MO    NE    NY    NC    ND    OH    PA      SD    TX    WI               
2001    CO    GA    IL    IN    IA    KS    KY    MI    MN    MO    NE    NY    NC    ND    OH    PA      SD    TX    WI               
2005    CO    GA    IL    IN    IA    KS    KY    MI    MN    MO    NE    NY    NC    ND    OH    PA      SD    TX    WI               
2010    CO    GA    IL    IN    IA    KS    KY    MI    MN    MO    NE    NY    NC    ND    OH    PA      SD    TX    WI               
Soybeans   
1996    AR      IL    IN    IA        LA        MN    MS    MO    NE        OH        TN      WI           
1997    AR    DE    IL    IN    IA    KS    KY    LA      MI    MN    MS    MO    NE    NC      OH    PA    SD    TN      WI           
1998    AR      IL    IN    IA    KS    KY    LA      MI    MN    MS    MO    NE    NC      OH      SD    TN               
1999    AR      IL    IN    IA    KS    KY    LA      MI    MN    MS    MO    NE    NC      OH    PA    SD    TN               
2000    AR      IL    IN    IA    KS    KY    LA      MI    MN    MS    MO    NE    NC    ND    OH      SD    TN      WI           
2002    AR      IL    IN    IA    KS    KY    LA    MD    MI    MN    MS    MO    NE    NC    ND    OH      SD    TN    VA    WI           
2006    AR      IL    IN    IA    KS    KY    LA      MI    MN    MS    MO    NE    NC    ND    OH      SD    TN    VA    WI           
2012    AR      IL    IN    IA    KS    KY    LA      MI    MN    MS    MO    NE    NC    ND    OH      SD    TN    VA    WI           

Cotton

1996      AZ    AR    CA    GA    LA    MS          TN    TX                               
1997    AL    AZ    AR    CA    GA    LA    MS    MO    NC    SC    TN    TX                               
1998    AL    AZ    AR    CA    GA    LA    MS      NC      TN    TX                               
1999    AL    AZ    AR    CA    GA    LA    MS      NC      TN    TX                               
2000    AL    AZ    AR    CA    GA    LA    MS    MO    NC      TN    TX                               
2003    AL    AZ    AR    CA    GA    LA    MS    MO    NC    SC    TN    TX                               
2007    AL      AR    CA    GA    LA    MS    MO    NC    SC    TN    TX                               
Durum Wheat   
1996          ND                                               
1997          ND                                               
1998    CA      MT    ND    SD                                             
2000          ND                                               
2004        MT    ND                                               
2009      ID    MT    ND    SD                                             
Other Spring Wheat   
1996        MN    MT    ND                                             
1997        MN    MT    ND      SD                                         
1998      ID    MN    MT    ND    OR    SD    WA                                       
2000        MN    MT    ND      SD                                         
2004      ID    MN    MT    ND    OR    SD    WA                                       
2009    CO    ID    MN    MT    ND    OR    SD    WA                                       
Winter Wheat   
1996        CO    DE      ID        KS                MT    NE        OK    OR      SD    TX    WA     
1997        CO        ID    IL      KS              MO    MT    NE      OH    OK    OR    PA    SD    TX    WA     
1998      CA    CO      GA    ID    IL      KS      LA      MN    MS    MO    MT    NE    NC    OH    OK    OR      SD    TX    WA     
1999                  IN                                       
2000    AR      CO        ID    IL      KS    KY            MO    MT    NE    NC    OH    OK    OR      SD    TX    WA     
2004        CO        ID    IL      KS        MI        MO    MT    NE      OH    OK    OR      SD    TX    WA     
2009        CO        ID    IL      KS        MI    MN      MO    MT    NE    ND    OH    OK    OR      SD    TX    WA     
Peanuts   
1999    AL      GA      NC    TX                                           
2004    AL    FL    GA      NC    TX                                           
2013    AL    FL    GA    SC    NC    TX                                           
Potatoes   
1996     ID    ME            WA                                         
1997     ID    ME    MN    ND     OR      WA    WI                                       
1998              PA      WI                                       
Rice   
2000    AR    CA    LA    MS      TX                                           
2006    AR    CA    LA    MS    MO    TX                                           
2013    AR    CA    LA    MS    MO    TX                                           
Sugarbeets   
2000    CA    CO    ID    MI    MN    MT    NE    ND    OR    WA    WY                                 
Sunflowers   
1999    KS    ND    SD                                                 
Oats   
2005    IL    IA    KS    MI    MN    NE    NY    ND    PA    SD    TX    WI                               
Barley   
2003      CA      ID    MN    MT    ND      PA    SD    UT      WA    WI    WY                         
2011    AZ    CA    CO    ID    MN    MT    ND    OR    PA        VA    WA    WI    WY                         
Sorghum   
2003    CO    KS    MO    NE    OK    SD    TX                                         
2011    CO    KS      NE    OK    SD    TX                                         
Flue-cured Tobacco   
1996    GA    NC    SC    VA                                               
Tobacco   
2008    GA    KY    NC    SC    TN    VA                                           
Cow-calf   
1996        CA    CO    FL      ID    IA    IL    KS    KY    MO      MT    NE    NM    ND    OK    OR    SD    TN    TX      WY       
2008    AL    AR    CA    CO    FL    GA      IA      KS    KY    MO    MS    MT    NE    NM    ND    OK    OR    SD    TN    TX    VA    WY       
Hogs   
1998    AL    AR    CO    GA    IL    IN    IA    KS    KY    MI    MN    MO    NE    NC    OH    OK    SC      SD    TN    UT    VA    WI         
2004      AR    CO    GA    IL    IN    IA    KS    KY    MI    MN    MO    NE    NC    OH    OK      PA    SD        VA    WI         
2009      AR    CO    GA    IL    IN    IA    KS    KY    MI    MN    MO    NE    NC    OH    OK      PA    SD        VA    WI         
Dairy   
2000    AZ    CA      FL    GA    ID    IL    IN    IA      KY      MI    MN    MO    NM    NY    OH      PA    TN    TX    VT    VA    WA    WI   
2005    AZ    CA      FL    GA    ID    IL    IN    IA      KY      MI    MN    MO    NM    NY    OH      PA    TN    TX    VT    VA    WA    WI   
2010    AZ    CA    CO    FL    GA    ID    IL    IN    IA    KS    KY    ME    MI    MN    MO    NM    NY    OH    OR    PA    TN    TX    VT    VA    WA    WI   
Broilers   
2006    AL    AR    CA    DE    GA    KY    LA    MD    MO    MS    NC    OK    PA    SC    TN    TX    VA                     
2011    AL    AR    CA    DE    GA    KY    LA    MD    MO    MS    NC    OK    PA    SC    TN    TX    VA                     

Data Collection

ARMS data are collected by self-administered mailed questionnaires and personal interviews conducted by trained enumerators using several types of questionnaires. Mail responses can be returned on the printed form or through electronic data reporting (EDR). Data collection starts in June with Phase I, continues during the fall with Phase II (production practices and cost data), and finishes in late winter (after the close of the year) with Phase III (whole-farm income and costs data). Phase II data are collected at the individual field and production unit level, while Phase III is collected for the whole farm business. Survey instruments (available for download) show the range of questions asked for Phase II and Phase III on income, expenses, assets, debt, and specific crop/livestock management practices. Interviewers' manuals (also available for download) outline detailed enumeration procedures for each phase of the survey, providing specific instruction on how to conduct an interview and specific methods and examples on recording complicated data. Questionnaires include specific notes with each question to guide respondents.

Data Processing and Analysis

Data collection and preliminary editing/analysis and processing are managed by NASS Headquarters in Washington, D.C. and by 12 NASS regional offices throughout the contiguous U.S. About 200-300 NASS personnel as well as ERS personnel are involved in preliminary editing/analysis and processing prior to delivery of a raw survey file to ERS.

collectionflow.fw.png

Initial processing includes manual review and possible imputation by enumerators, supervisory enumerators, or regional statisticians for missing items or items that are not allowed to be initially coded as a nonresponse (-1). NASS then uses a set of automated processing tools, including FEITH, which provides scanned images of all questionnaires; PRISM2, an interactive data review system; and IDAS (Interactive Data Analysis System) for data review at the individual (respondent) and aggregated (strata, State, etc.) data levels.

Additional post-survey processing includes computer imputation to replace nonresponse codes for a number of items, survey weight calibration, outlier adjustment, and final summarization. ERS conditions the final raw survey data file with additional data editing, analysis, item imputation, and variable creation for use in research.

Editing

Survey enumerators, supervisory enumerators, and NASS regional office personnel provide a preliminary edit of completed ARMS questionnaires before entry into the NASS computer system. The questionnaires are then scanned into FEITH and data are manually keyed before being processed with a computerized edit program used to identify potential data errors. NASS staff in Washington and in regional offices, assisted by ERS staff, then check for errors and consistency using the PRISM2 editing program. After further analysis, NASS passes a completed raw survey file to ERS. ERS then creates a final dataset that includes some retained NASS variables, all keyed data, and several hundred additional calculated variables covering farm operation and farm operator household characteristics. ERS performs a final round of review, analysis, and updates of the complete ARMS research file prior to its release.

NASS Computer Edit/Analysis

The NASS computer edit identifies potential errors involving known physical relationships, obvious coding errors, and basic economic relationships between interrelated questionnaire items. Specifications for the computer edit are reviewed and updated annually by ERS and NASS. Errors/potential errors are divided into two categories—critical errors (the item value must be corrected or a comment explaining the value must be recorded), and warnings (the item value should be reviewed and/or updated).

NASS uses a combination of software tools including PRISM, FEITH, and IDAS to examine and evaluate ARMS data during the editing phase. IDAS is the principal tool for examination of individual report data. IDAS provides tabular and graphical displays of survey-level report items and additional analysis variables by State, region, and at the U.S. level. The objective is to facilitate the identification of remaining data errors. Custom Hyperion IR queries are also used if specific issues arise during analysis. Both ERS and NASS personnel participate in the editing and analysis of ARMS data with IDAS.

NASS Imputation

Survey responses are divided into two categories, those for which a value must be provided (either by the respondent or manually imputed by a NASS statistician) and those that can be initially coded as a nonresponse (the respondent refused to provide or did not know the answer). Items for which a value must be initially provided constitute a relatively small share (typically less than 30 percent) of all items. Statisticians are instructed to manually impute values for these items using one or more methods/sources including relationships in the questionnaire, similar reports, other State surveys, other outside sources and publications, and data from the ERS website.

Most of the items on the ARMS Phase III survey can be initially coded as a nonresponse (the respondent did not know or refused to provide an answer to an item). In 2005, for example, about 75 percent of all items fell in this category. At the request of ERS, NASS uses a computer algorithm to impute data for a small subset of these items (144 in 2011)—typically, about 10 percent of all items that can be coded as nonresponses. The items selected by ERS are based on its mission requirements for time sensitive development of farm operation/farm operator household financial and structural characteristics and by NASS’s need to complete analysis and the Farm Production Expenditures Summary publication. The remaining missing items are coded with a value of -1 in the file delivered to ERS.

The NASS imputation algorithm computes unweighted mean values for donors (farms reporting a positive value for an item) in a specific set of farm groupings based on location, farm size, and farm type. Donors with “extreme values” are excluded from calculations. In assigning an imputed value, the algorithm uses the first grouping (starting with group 1) that contains at least 10 donors. In 2012, 80 percent of imputed values were obtained from group 1. Means are computed for 12 imputation groups:

  1. Region, Sales Class, Farm Type
  2. Region, Pooled Sales Class, Farm Type
  3. U.S., Sales Class, Farm Type
  4. U.S., Pooled Sales Class, Farm Type
  5. Region, Sales Class, Pooled Farm Type
  6. Region, Pooled Sales Class, Pooled Farm Type
  7. U.S., Sales Class, Pooled Farm Type
  8. U.S., Pooled Sales Class, Pooled Farm Type
  9. Region, Farm Type
  10. Region, Sales Class
  11. Region Average
  12. U.S. Average

Farm depreciation expense, landlord taxes, and other assets are examples of individual items most frequently imputed. The remaining items for which data are most commonly imputed can be grouped into three categories: farm labor, other farm-related income, and farm assets.

Calibration

After editing and imputation, survey sample weights are calibrated so that weighted survey totals for selected items match official USDA estimates for production or acreage where possible. A decision rule is employed to exclude calibration targets for survey items whose weighted totals fall below or above predetermined thresholds in relation to official estimates. The calibration items in the last few years have numbered between 30-32 targets as follows:

  • Number of Farms by 8 economic classes
  • Number of Farms by nonestimate States
  • Number of Farms - Total
  • Acres - Corn for grain
  • Acres - Wheat for grain acres
  • Acres - Soybean acres
  • Acres - Cotton acres
  • Acres - Vegetable acres
  • Acres - Fruit acres
  • Acres - Hay acres
  • Acres - Peanut acres
  • Acres - Rice acres
  • Acres - Sugarcane/sugarbeet acres
  • Acres - Tobacco acres
  • Acres - Nursery/floriculture (2 targets)
  • Acres - Costs of production crop target added that year for each crop not listed above
  • Cattle
  • Cattle on feed
  • Hogs and pigs
  • Milk
  • Eggs
  • Broilers
  • Turkeys

Outlier Process

Following calibration, outliers are identified and reviewed by the official USDA-NASS National Outlier Review Board. There are typically five or fewer national outliers each year. An outlier is defined as a sampled farm where weighted (expanded) data for total expenses accounts for 0.5 percent of U.S. total expenses and/or 2.5 percent of regional total expenses. The review board usually adjusts outlier sample weights downward. The general rule for treating outliers is to reweight to the median weight of the matching economic class by farm type. Following the National Outlier Review Board, outlier weights are frozen at board levels and the calibrated summary is rerun. Following recalibration, survey weights are rechecked for the possible introduction of new outliers. An iterative procedure is used to adjust the weights of the new outliers, and recalibrate the weights until outliers are mitigated.

ERS Dataset Conditioning

Following receipt of the raw survey file from NASS, ERS creates a preliminary version of the final ERS research file. The majority of NASS calculated variables are removed mainly because they cannot be updated if changes are made. Conditioning programs calculate several hundred chronologically and methodologically consistent financial and operational variables and add them to the raw survey data file. The additional research variables are typically calculated from a combination of raw survey items. ERS adds geographic identifiers (such as ERS farm resource regions), demographic information (for example, categorical variables based on the U.S. 2010 Population Census), and other county, regional, and national economic information. ARMS III data files are available for 1996 to present.

ERS Imputation

In creating the final research file, ERS imputes data for about 40 items that are necessary to meet time-sensitive ERS mission requirements for the publication of farm operation and farm operator household data. This includes farm operator debt and items pertaining to the farm operator and the farm operator’s household. Collectively, ERS and NASS impute data for about 15 percent of all items that could be coded as nonresponses. The remaining items are retained with a “-1” code in the final ERS research file.

In general, ERS employs the same basic methodology as NASS, using mean values for responding farms to replace refusal codes. ERS imputation schemes can be procedurally complex incorporating other survey items and varying in some cases by questionnaire version and the level of item aggregation. ERS imputes values for a number of items related to the farm operator and operator’s household, including:

  • Off-farm income items—wages and salaries, net income from other farms, net income from other business, net income from renting farm land, interest income, dividend income, capital gains/loss, private retirement income, public retirement income, and other
  • Share of farm income retained by operator household
  • Farm operation debt
  • Operator household size
  • Household expenses, nonfarm assets, and nonfarm debt
  • Total value farm sales, net operating income, and total off-farm income previous year.

ERS uses a larger set of classification variables (in addition to farm size, type, and location) in imputing household items, including the number of operators, the operators’ age class, education level, marital status, and retirement status, as well as the farms’ legal organization. Differences in questionnaire versions are also incorporated into imputation methods for the farm operator’s household.

ERS Data Review

Following creation of the preliminary version of the research data file, ERS performs a final review of the data. Key financial and operational values and relationships are audited for internal (within record) and external (across records and across time) consistency. SAS programs are used to identify errors in the computer code used to create the final research file and remaining problems in the data that were not identified during the NASS edit/analysis. Some records are updated by the conditioning programs while others are researched again at NASS and updates applied if errors are found. In evaluating potential data edits ERS also uses the auxiliary comment file provided by NASS. The comment file includes explanations at the survey record (farm) level by ERS and NASS record analysts about enumerator comments or “unusual data values” remaining in the survey file.

Quality Control

ERS and NASS provide "train the trainer" workshops prior to data collection for each survey. Regional and State statisticians then train enumerators through a series of dispersed workshops. ERS and NASS develop and provide training materials to the State survey statisticians. After questionnaires are completed by the enumerators, each questionnaire is reviewed by supervisory enumerators for completeness, inconsistent responses, or errors, and then transferred to a NASS State or Regional office where each questionnaire is reviewed before it is keyed into an electronic format. While the survey database is assembled, a computer edit is used to identify potential recording errors or inconsistencies in data relationships (for example, interest expense matched with farm debt). In the process of conditioning the survey database, uncharacteristic responses are investigated and data are verified or corrected. Select cells can be marked for later computer imputation. The Interviewer’s manual and the Survey Administration manual are updated each year and provide specific details to provide uniform treatment of data collection and data processing procedures.

Changes in Methodology

Time period    Change   
2013    The survey was redesigned to reduce expenditures while maintaining sample size. The Core version of ARMS Phase III was eliminated, and questionnaire length in other Phase III versions was reduced. Remaining versions shifted to mail mode, while retaining enumerator assistance on request.   
2012    The ARMS questionnaire was integrated with the Census of Agriculture; ARMS questionnaires included all census questions, and primary ARMS questions.   
2010-present    New top-end value codes were introduced for other farm assets and off-farm income, assets, and debt, replacing the top code of 1.5 million and above that had been in place since 1996. The new top code is 10 million and above, and 4 additional codes were introduced to cover values between 1.5 million and 9.99 million.   
2007    The ARMS questionnaire was integrated with the Census of Agriculture; ARMS questionnaires included all census questions, and primary ARMS questions.   
2003-2012    A new, shorter, core version of the survey was introduced, as a mail-out survey with enumerator follow-up upon request. The added sample size allowed for State-level estimates to be generated for 16 major agricultural States. The number of useable responses rose to 20,000 or more after core introduction, compared to 10,000 before.   
2002    The ARMS questionnaire was integrated with the Census of Agriculture; ARMS questionnaires included most census questions, and primary ARMS questions.   

Variance Estimation

ERS and NASS follow strict statistical procedures when designing, collecting, and summarizing ARMS data. The observations from any given sample of the population must be weighted properly to ensure that sample estimates of totals and averages are representative of the total farm population. Through a process known as calibration survey weights are adjusted so that sample estimates of population totals hit known targets such as the total acres of corn.

Estimating variances of sample estimates gives an indication of how an estimate may vary if a different ARMS sample were drawn. An estimate’s variance gives a sense of how close a sample estimate is likely to be to the true population value. Given the complex design of ARMS, a specific method for estimating variances, known as the Delete-a-Group Jackknife, is recommended for calculating the variance of a sample estimate.

The following documents include information on the calibration of sample weights and variance estimation.

  • Variance Estimation with USDA’s Farm Costs and Returns Surveys and Agricultural Resource Management Study Surveys. This paper is an overview of survey estimators, sample design, hypothesis testing, disclosure rules, and reliability measures for ARMS followed by statistical program documentation. Sums, ratios, means, multiple regression, binomial logit analysis, and order statistics are covered. Contact Robert Dubman for a copy of this paper.

Nonresponse

As with most surveys, some sampled farm operators do not respond to particular questions (item nonresponse) or to the entire survey (unit nonresponse). For some questions, item nonresponse is addressed by imputation, which is a statistical procedure that uses information from other questions to fill in missing values. Unit nonresponse, in contrast, is at least partially addressed through adjustments to sample weights, including calibration, that help to account for the missing farms.

The following publications provide information such as patterns in nonresponse and potential consequences and solutions.

Technical Documents

Several documents are available which explain in technical detail various issues related to the ARMS.

  • Farm Production Expenditures Methodology and Quality Measures, PDF icon (16x16) 2013.This document explains the methodology and quality measures for NASS estimates of farm production expenditures, covering topics such as sampling, data collection methods, nonresponse adjustments, calibration, and outliers.
  • An Economist’s Primer on Survey Samples, PDF icon (16x16) 2000. This document provides an accessible overview of how the implications of incorporation sample design into economic analysis of survey data.

Maps

This set of maps helps data users visualize and better understand the geographical scope and level of aggregation by which many ARMS data are summarized. For example, the ARMS web data tool presents farm financial estimates aggregated to Farm Resource Regions, which do not follow State boundaries. Maps below demonstrate the different ways data are often summarized.

U.S. Farm Resource Regions: Farm resource regions are defined using farm production regions, land resource regions, crop reporting districts, and farm characteristics. The regions are designated at the county level and therefore do not generally follow State boundaries. See Farm Resource Regions, AIB-760, August 2000.

Farm Resource Regions

Farm Production Regions: The older Farm Production Regions, in following State boundaries, group unlike areas together because a single State often encompasses different soils and typography. For example, the old Appalachian Region, comprised of Tennessee, Kentucky, North Carolina, West Virginia, and Virginia, contains the Appalachian Mountains, Piedmont, and Coastal Plain areas, all of which have quite different agriculture.

Farm Production Regions

Patterns of Agricultural Diversity: County clusters, based on types of commodities produced, have shown that a few commodities tend to dominate farm production in specific geographic areas that cut across State boundaries. The climate, soil, water, and topography in localized geographic areas tend to constrain the types of crops and livestock that will thrive there. See Diversity in U.S. Agriculture: A New Delineation by Farming Characteristics PDF icon (16x16) , AER-646, July 1991, for more information.

Cluster analysis of farm characteristics

USDA Land Resource Regions: In constructing the ERS production regions, analysts identified where areas with similar types of farms intersected with areas of similar physiographic, soil, and climatic traits, as reflected in USDA's Land Resource Regions.

USDA land resource regions

NASS Crop Reporting Districts: County reporting districts influenced the construction of the ERS farm resource regions by conforming intersecting areas to follow the boundaries of NASS Crop Reporting Districts (CRD), which are aggregates of counties. With more and more data available at the county level, geographic representations need no longer be constrained to follow State boundaries.

NASS crop reporting districts

NASS ARMS Regions: Five Farm Production Expenditure Regions

Other Resources

Last updated: Tuesday, December 02, 2014

For more information contact: Mitch Morehart

Text Only Options

Top of page


Text Only Options

Open the original version of this page.

Usablenet Assistive is a UsableNet product. Usablenet Assistive Main Page.