The Agricultural Resource Management Survey (ARMS) is sponsored jointly by USDA's Economic Research Service (ERS) and National Agricultural Statistics Service (NASS). ARMS was first conducted in 1996. It combined USDA's previous cropping practices, chemical use, and farm costs and returns surveys, which were conducted separately from 1975 to 1995. ARMS is a multiphase series of interviews with farm operators about their cropping practices, farm businesses, and households.
Farmers and ranchers selected for the survey are contacted to verify that they still qualify as a farm and that they produce the specific commodities targeted by Phase II that year.
Phase II questionnaires are only directed to operations producing the survey year’s target crop(s). The target crop is rotated on a multiyear cycle. Data are periodically collected for each major commodity, e.g. corn producers were surveyed in 2001, 2005, and 2010. Common information collected for all surveyed commodities includes (but is not limited to):
Beginning in 2012, the Phase III uses only two questionnaire versions: a general Costs and Returns Report and an expanded Costs and Returns Report for target crop or livestock producers, which collects detailed commodity-specific information. Prior to 2012, a Core (short) version of the questionnaire was used to simplify the general Costs and Returns Report. Information collected on all versions includes (but is not limited to):
ARMS is a series of interviews with farm operators about their management practices, farm business structure and finances, and household characteristics. It is conducted annually in three phases:
Diagram of ARMS modular design that connects field practices, financial data, and household demographics
NASS uses two sampling frames to sample a total of about 30,000 farm and ranch operations for ARMS each year. The majority of farms—around 94 percent—come from the List Frame.
Strata are divisions within the sample frames that group farms by region (or by State in the case of 15 heavily sampled States), farm sales category, and commodity specialization. Farms in different strata are sampled with a different probability of selection to ensure that each sample includes a sufficient number of farms in different regions, sizes, and commodities. Within a stratum, the weight (expansion factor) is based on the probability of each sampled unit’s selection.
Because the ARMS sample is not a simple random sample, each observation has a different weight, or expansion factor, to reflect its probability of selection and, therefore, what part of the sampled universe it represents. Appropriate sample weights (expansion factors) are provided to prepare population estimates from the survey results. Population estimates are constructed by weighting each sample with the appropriate expansion factor. A jackknife re-sampling process is used with 30 additional weights from NASS for each sample to estimate the variances for each data item.
Furthermore, data from the Phase II of ARMS is divided into three data files: 1) fertilizers, 2) pesticides, and 3) all other data designated as the main file (e.g., field characteristics, management practices, and production input data other than fertilizers and pesticides). Sample weights associated with each of the three data files depend on the number of usable responses for the respective parts of the Phase II questionnaire. The usability of these tables for the construction of chemical or fertilizer use estimates is determined independently from the completion of the remainder of the questionnaire. Typically, slightly different response rates exist for these three parts of the questionnaire, and hence, weights differ between the main file and the two subfiles (pesticide and fertilizer). Cross-tabbing of variables across the three data files can give different population estimates for the same variable. In general, such population estimate differences across tables are relatively small. For more detail, see Farm Production Expenditures Methodology and Quality Measures.
Questionnaires are printed forms or computer programs used to ask specific questions and record the responses given by the sampled respondents. Due to the complex design of the ARMS survey, multiple questionnaires are used each year. Phase I questionnaires are used to collect current information for accurate sampling. All complete Phase II respondents are asked to complete a Phase III follow-on report. Every effort is made to ensure that both the Phase II and Phase III questionnaires are completed for these operations in both samples. Data from both phases provide the link between agricultural resource use and farm financial conditions, one of the cornerstones of the ARMS design. ERS and NASS jointly develop the questionnaires.
Manuals are produced each year to instruct enumerators and editors, and provide survey documentation. Both questionnaires and manuals can be found through visiting the ARMS questionnaires and manuals page.
|2 = Phase II field-level Production Practices Report only.
2,3 = Both Phase II field-level Production Practices Report and Phase III whole-farm Costs of Production survey.
3 = Phase III whole-farm Costs of Production survey only.
4 = Tenure, Ownership, and Transition of Agricultural Land Survey (TOTAL).
|Other Spring wheat|
|Tenure, ownership, and transition of Agricultural Land Survey (Total)|
ARMS data are collected by self-administered mailed questionnaires and personal interviews conducted by trained enumerators using several types of questionnaires. Mail responses can be returned on the printed form or through electronic data reporting (EDR). Data collection starts in June with Phase I, continues during the fall with Phase II (production practices and cost data), and finishes in late winter (after the close of the year) with Phase III (whole-farm income and costs data). Phase II data are collected at the individual field and production unit level, while Phase III is collected for the whole farm business. Survey instruments (available for download) show the range of questions asked for Phase II and Phase III on income, expenses, assets, debt, and specific crop/livestock management practices. Interviewers' manuals (also available for download) outline detailed enumeration procedures for each phase of the survey, providing specific instruction on how to conduct an interview and specific methods and examples on recording complicated data. Questionnaires include specific notes with each question to guide respondents.
Data collection and preliminary editing/analysis and processing are managed by NASS Headquarters in Washington, D.C. and by 12 NASS regional offices throughout the contiguous U.S. About 200-300 NASS personnel as well as ERS personnel are involved in preliminary editing/analysis and processing prior to delivery of a raw survey file to ERS.
Flow chart shows the steps involved in collecting, processing, and delivering ARMS data used by researchers and policy analysts
Initial processing includes manual review and possible imputation by enumerators, supervisory enumerators, or regional statisticians for missing items or items that are not allowed to be initially coded as a nonresponse (-1). NASS then uses a set of automated processing tools, including FEITH, which provides scanned images of all questionnaires; PRISM2, an interactive data review system; and IDAS (Interactive Data Analysis System) for data review at the individual (respondent) and aggregated (strata, State, etc.) data levels.
Additional post-survey processing includes computer imputation to replace nonresponse codes for a number of items, survey weight calibration, outlier adjustment, and final summarization. ERS conditions the final raw survey data file with additional data editing, analysis, item imputation, and variable creation for use in research.
Survey enumerators, supervisory enumerators, and NASS regional office personnel provide a preliminary edit of completed ARMS questionnaires before entry into the NASS computer system. The questionnaires are then scanned into FEITH and data are manually keyed before being processed with a computerized edit program used to identify potential data errors. NASS staff in Washington and in regional offices, assisted by ERS staff, then check for errors and consistency using the PRISM2 editing program. After further analysis, NASS passes a completed raw survey file to ERS. ERS then creates a final dataset that includes some retained NASS variables, all keyed data, and several hundred additional calculated variables covering farm operation and farm operator household characteristics. ERS performs a final round of review, analysis, and updates of the complete ARMS research file prior to its release.
The NASS computer edit identifies potential errors involving known physical relationships, obvious coding errors, and basic economic relationships between interrelated questionnaire items. Specifications for the computer edit are reviewed and updated annually by ERS and NASS. Errors/potential errors are divided into two categories—critical errors (the item value must be corrected or a comment explaining the value must be recorded), and warnings (the item value should be reviewed and/or updated).
NASS uses a combination of software tools including PRISM, FEITH, and IDAS to examine and evaluate ARMS data during the editing phase. IDAS is the principal tool for examination of individual report data. IDAS provides tabular and graphical displays of survey-level report items and additional analysis variables by State, region, and at the U.S. level. The objective is to facilitate the identification of remaining data errors. Custom Hyperion IR queries are also used if specific issues arise during analysis. Both ERS and NASS personnel participate in the editing and analysis of ARMS data with IDAS.
Survey responses are divided into two categories, those for which a value must be provided (either by the respondent or manually imputed by a NASS statistician) and those that can be initially coded as a nonresponse (the respondent refused to provide or did not know the answer). Items for which a value must be initially provided constitute a relatively small share (typically less than 30 percent) of all items. Statisticians are instructed to manually impute values for these items using one or more methods/sources including relationships in the questionnaire, similar reports, other State surveys, other outside sources and publications, and data from the ERS website.
Most of the items on the ARMS Phase III survey can be initially coded as a nonresponse (the respondent did not know or refused to provide an answer to an item). In 2005, for example, about 75 percent of all items fell in this category. At the request of ERS, NASS uses a computer algorithm to impute data for a small subset of these items (144 in 2011)—typically, about 10 percent of all items that can be coded as nonresponses. The items selected by ERS are based on its mission requirements for time sensitive development of farm operation/farm operator household financial and structural characteristics and by NASS’s need to complete analysis and the Farm Production Expenditures Summary publication. The remaining missing items are coded with a value of -1 in the file delivered to ERS.
Beginning in 2014, item level imputations are now done using a multivariate approach. Prior to the implementation of the multivariate approach, NASS used an un-weighted conditional means imputation system that placed records into homogenous groups and imputed based off of reported data from those groups. The new multivariate approach uses a regression-based technique that allows for flexibility in the selection of conditional models while providing a valid joint distribution. In this procedure, labeled as Iterative Sequential Regression (ISR), parameter estimates and imputations are obtained using a Markov chain Monte Carlo sampling method. Using ISR, we are better able to preserve the relationships within the data and also allow the imputed values to better represent the variability of the data. Records with imputed data are re-edited to ensure the returned value is acceptable. The imputation algorithm delivers an acceptable value more than 99.5 percent of the time and Field Office statisticians are required to manually impute for any missing items.
Before 2014, the NASS imputation algorithm computed unweighted mean values for donors (farms reporting a positive value for an item) in a specific set of farm groupings based on location, farm size, and farm type. Donors with “extreme values” are excluded from calculations. In assigning an imputed value, the algorithm uses the first grouping (starting with group 1) that contains at least 10 donors. In 2012, 80 percent of imputed values were obtained from group 1. Means are computed for 12 imputation groups:
Farm depreciation expense, landlord taxes, and other assets are examples of individual items most frequently imputed. The remaining items for which data are most commonly imputed can be grouped into three categories: farm labor, other farm-related income, and farm assets.
After editing and imputation, survey sample weights are calibrated so that weighted survey totals for selected items match official USDA estimates for production or acreage where possible. A decision rule is employed to exclude calibration targets for survey items whose weighted totals fall below or above predetermined thresholds in relation to official estimates. Targets are added in the year when commodities of interest are over-sampled if they are not already part of the targets listed below.
The calibration items in the last few years have numbered between 30-32 targets as follows:
Following calibration, outliers are identified and reviewed by the official USDA-NASS National Outlier Review Board. There are typically five or fewer national outliers each year. An outlier is defined as a sampled farm where weighted (expanded) data for total expenses accounts for 0.5 percent of U.S. total expenses and/or 2.5 percent of regional total expenses. The review board usually adjusts outlier sample weights downward. The general rule for treating outliers is to reweight to the median weight of the matching economic class by farm type. Following the National Outlier Review Board, outlier weights are frozen at board levels and the calibrated summary is rerun. Following recalibration, survey weights are rechecked for the possible introduction of new outliers. An iterative procedure is used to adjust the weights of the new outliers, and recalibrate the weights until outliers are mitigated.
Following receipt of the raw survey file from NASS, ERS creates a preliminary version of the final ERS research file. The majority of NASS calculated variables are removed mainly because they cannot be updated if changes are made. Conditioning programs calculate several hundred chronologically and methodologically consistent financial and operational variables and add them to the raw survey data file. The additional research variables are typically calculated from a combination of raw survey items. ERS adds geographic identifiers (such as ERS farm resource regions), demographic information (for example, categorical variables based on the U.S. 2010 Population Census), and other county, regional, and national economic information. ARMS III data files are available for 1996 to present.
In creating the final research file, ERS imputes data for about 40 items that are necessary to meet time-sensitive ERS mission requirements for the publication of farm operation and farm operator household data. This includes farm operator debt and items pertaining to the farm operator and the farm operator’s household. Collectively, ERS and NASS impute data for about 15 percent of all items that could be coded as nonresponses. The remaining items are retained with a “-1” code in the final ERS research file.
In general, ERS employs the same basic methodology as NASS, using mean values for responding farms to replace refusal codes. ERS imputation schemes can be procedurally complex incorporating other survey items and varying in some cases by questionnaire version and the level of item aggregation. ERS imputes values for a number of items related to the farm operator and operator’s household, including:
ERS uses a larger set of classification variables (in addition to farm size, type, and location) in imputing household items, including the number of operators, the operators’ age class, education level, marital status, and retirement status, as well as the farms’ legal organization. Differences in questionnaire versions are also incorporated into imputation methods for the farm operator’s household.
Following creation of the preliminary version of the research data file, ERS performs a final review of the data. Key financial and operational values and relationships are audited for internal (within record) and external (across records and across time) consistency. SAS programs are used to identify errors in the computer code used to create the final research file and remaining problems in the data that were not identified during the NASS edit/analysis. Some records are updated by the conditioning programs while others are researched again at NASS and updates applied if errors are found. In evaluating potential data edits ERS also uses the auxiliary comment file provided by NASS. The comment file includes explanations at the survey record (farm) level by ERS and NASS record analysts about enumerator comments or “unusual data values” remaining in the survey file.
ERS and NASS provide "train the trainer" workshops prior to data collection for each survey. Regional and State statisticians then train enumerators through a series of dispersed workshops. ERS and NASS develop and provide training materials to the State survey statisticians. After questionnaires are completed by the enumerators, each questionnaire is reviewed by supervisory enumerators for completeness, inconsistent responses, or errors, and then transferred to a NASS State or Regional office where each questionnaire is reviewed before it is keyed into an electronic format. While the survey database is assembled, a computer edit is used to identify potential recording errors or inconsistencies in data relationships (for example, interest expense matched with farm debt). In the process of conditioning the survey database, uncharacteristic responses are investigated and data are verified or corrected. Select cells can be marked for later computer imputation. The Interviewer’s manual and the Survey Administration manual are updated each year and provide specific details to provide uniform treatment of data collection and data processing procedures.
|2013||The survey was redesigned to reduce expenditures while maintaining sample size. The Core version of ARMS Phase III was eliminated, and questionnaire length in other Phase III versions was reduced. Remaining versions shifted to mail mode, while retaining enumerator assistance on request.|
|2012||The ARMS questionnaire was integrated with the Census of Agriculture; ARMS questionnaires included all census questions, and primary ARMS questions.|
|2010–present||New top-end value codes were introduced for other farm assets and off-farm income, assets, and debt, replacing the top code of 1.5 million and above that had been in place since 1996. The new top code is 10 million and above, and 4 additional codes were introduced to cover values between 1.5 million and 9.99 million.|
|2007||The ARMS questionnaire was integrated with the Census of Agriculture; ARMS questionnaires included all census questions, and primary ARMS questions.|
|2003–2012||A new, shorter, core version of the survey was introduced, as a mail-out survey with enumerator follow-up upon request. The added sample size allowed for State-level estimates to be generated for 16 major agricultural States. The number of useable responses rose to 20,000 or more after core introduction, compared to 10,000 before.|
|2002||The ARMS questionnaire was integrated with the Census of Agriculture; ARMS questionnaires included most census questions, and primary ARMS questions.|
ERS and NASS follow strict statistical procedures when designing, collecting, and summarizing ARMS data. The observations from any given sample of the population must be weighted properly to ensure that sample estimates of totals and averages are representative of the total farm population. Through a process known as calibration survey weights are adjusted so that sample estimates of population totals hit known targets such as the total acres of corn.
Estimating variances of sample estimates gives an indication of how an estimate may vary if a different ARMS sample were drawn. An estimate’s variance gives a sense of how close a sample estimate is likely to be to the true population value. Given the complex design of ARMS, a specific method for estimating variances, known as the Delete-a-Group Jackknife, is recommended for calculating the variance of a sample estimate.
The following documents include information on the calibration of sample weights and variance estimation.
As with most surveys, some sampled farm operators do not respond to particular questions (item nonresponse) or to the entire survey (unit nonresponse). For some questions, item nonresponse is addressed by imputation, which is a statistical procedure that uses information from other questions to fill in missing values. Unit nonresponse, in contrast, is at least partially addressed through adjustments to sample weights, including calibration, that help to account for the missing farms.
The following publications provide information such as patterns in nonresponse and potential consequences and solutions.
Several documents are available which explain in technical detail various issues related to the ARMS.
This set of maps helps data users visualize and better understand the geographical scope and level of aggregation by which many ARMS data are summarized. For example, the ARMS web data tool presents farm financial estimates aggregated to Farm Resource Regions, which do not follow State boundaries. Maps below demonstrate the different ways data are often summarized.
U.S. Farm Resource Regions: Farm resource regions are defined using farm production regions, land resource regions, crop reporting districts, and farm characteristics. The regions are designated at the county level and therefore do not generally follow State boundaries. See the ERS report, Farm Resource Regions, (AIB-760, August 2000), for more information, or download the county-to-ERS Resource Region aggregation in Excel.
A map of U.S. Farm Resource Regions
Farm Production Regions: The older Farm Production Regions, in following State boundaries, group unlike areas together because a single State often encompasses different soils and typography. For example, the old Appalachian Region, comprised of Tennessee, Kentucky, North Carolina, West Virginia, and Virginia, contains the Appalachian Mountains, Piedmont, and Coastal Plain areas, all of which have quite different agriculture.
A map of Farm Production Regions
Patterns of Agricultural Diversity: County clusters, based on types of commodities produced, have shown that a few commodities tend to dominate farm production in specific geographic areas that cut across State boundaries. The climate, soil, water, and topography in localized geographic areas tend to constrain the types of crops and livestock that will thrive there. See Diversity in U.S. Agriculture: A New Delineation by Farming Characteristics, AER-646, July 1991, for more information.
USDA Land Resource Regions: In constructing the ERS production regions, analysts identified where areas with similar types of farms intersected with areas of similar physiographic, soil, and climatic traits, as reflected in USDA's Land Resource Regions.
A map of USDA Land Resource Regions
NASS Crop Reporting Districts: County reporting districts influenced the construction of the ERS farm resource regions by conforming intersecting areas to follow the boundaries of NASS Crop Reporting Districts (CRD), which are aggregates of counties. With more and more data available at the county level, geographic representations need no longer be constrained to follow State boundaries.
A map of NASS Crop Reporting Districts
NASS ARMS Regions: Five Farm Production Expenditure Regions
Last updated: Wednesday, October 25, 2017
For more information contact: Jeffrey W. Hopkins
Open the original version of this page.