Field Data Collection Process Development Considerations

1 INTRODUCTION

After you have identified the categories of data to be collected in the field (see Defining Data Categories and Collection Methods Fact Sheet), you should design an effective field data collection process. Understanding and establishing a good field data collection process can reveal problems and efficiencies and provide a detailed data flow process. The following sections detail general field data collection process design considerations, considerations for choosing the right data collection solution, and specific design considerations for both paper and digital field forms. Figure 1 highlights some of the considerations described from a layout perspective. When designing a field data collection process, you may build generic forms for specific tasks (for example, a generic groundwater sampling form) that may accommodate a variety of projects, or custom forms that concentrate on the specific field task at hand (for example, site-specific groundwater sampling forms tailored for low-flow sampling using installed dedicated pumps). Map out your field data collection project, capitalize on strengths, and look for gaps or deficiencies that should be addressed. Implementing the following best practices for field data collection process development will help eliminate potential field data collection problems, ensuring field data correctness, completeness, and consistency needed to ensure the overall quality of the data collected in the field (see also Field Data Collection Quality Assurance and Quality Control [QA/QC] Fact Sheet).

2 GENERAL FIELD DATA COLLECTION PROCESS DESIGN CONSIDERATIONS

Some essential steps that should be considered when designing field data collection processes are described below. These steps loosely correlate to the phases of the environmental data lifecycle (see Data Lifecycle Fact Sheet).

1. Define the purpose. Start by determining why you need to collect data. Consider how the data will be used once collected and define a clear purpose for collecting the data. 
2. Define the scope. Is the goal to design generic forms for specific tasks or custom forms? With many data collection solution combinations available, there may not be a single solution that will fit all projects. Different tools offer different features. However, managing multiple forms can require significant effort and present problems with data quality. The use of generic forms for multiple projects that have similar tasks can provide consistency in data management across projects with minimal effort. 
3. Define the metadata that should be captured. Consider what you might need to know in the future about how the information was captured, such as device type/identifier used for readings, software version being used for digital field forms, coordinate system, name of collector, date/time (if not one of the data elements), etc.
4. Define the data that you must collect to meet the purpose. Data are the individual pieces of information that you need to gather (i.e., unique sample ID, date/time of collection, latitude, longitude, elevation, readings, levels, etc.). Avoid collecting data that are superfluous or not critical for the purpose. Extra data may complicate processes and can increase labor in the field. 
5. Design for the data. For each piece of data to be collected, establish the following:
   • a. The data type (Boolean, date, integer, other numeric, text, etc.) 
   • b. The data-associated units, if applicable.
   • c. The possible valid values or range of values (for example, 0–14 for pH; true or false for any Booleans; low-flow, fixed volume, or no purge for an option group of purge method; pH, temperature, conductivity, and turbidity for a multiple-selection group or field parameters, etc.) 
   • d. Verification rules. Determine how and if information collected in the field will be verified. Will values be compared to an expected range or to previous readings? Are some data required only dependent on values? If so, document those rules for each element. For example, during an indoor air building survey, checking a box “cleaning agents observed” requires the user to enter the names of the observed cleaning agents. 
6. Order of operations. Consider the flow of data collection in the field. Which data will be generated at the same time? Considering this flow will ensure that the collection methods (paper or digital) can be established in a way that makes collection simplest for the field team. 
7. Define rules for construction of identifiers. These rules may define fields within identifiers, valid values for those fields, use (or non-use) of dashes and other special characters or digits. If identifiers may contain digits, the use of letters that may be mistaken for digits (O, B, I, C, Z) should be prohibited. 
8. Define quality assurance checks. Will data be checked daily or more frequently? 
9. Define backup procedures. How will digital data be backed up and with what frequency? Will paper field notes be taken in addition to digital forms in case of damage to electronic devices in the field? If paper forms are used, will scans or photos of paper forms be taken and how will those scans be backed up?
10. Define where the data will be stored. If paper forms are used, will they be stored as paper, will they be transcribed, or will they be scanned or photographed to be used and stored digitally? Will data be stored in an environmental data management system (EDMS) (see Environmental Data Management Systems White Paper)?
11. Define how the data will be used. Will the data be reported independently or used in combination with other data? 
12. Plan for improvement. Field data collection processes are rarely perfect the first time that they are used, so plan to evaluate the process after use and establish a process for improvement.

3 CHOOSE THE RIGHT DATA COLLECTION SOLUTION (PAPER OR DIGITAL)

After identifying your data requirements, selecting the right platform—paper or digital—can be a challenge. Determine environmental factors or site conditions that could affect your field data collection solution. 

Management of paper forms on a large field project, where each field technician is filling out a form, can be a formidable task because all forms must be collected, collated, recorded, and analyzed. Often, field staff will skip form headers to be filled in later. This can result in loss of data or errors in data. One way to mitigate this is to prepopulate known information to help reduce losses in data and improve efficiency in the field. Paper forms can also be lost or destroyed, which creates problems in the future if data need to be readdressed. However, sometimes pen and paper data collection may be the only viable option for certain organizations or is truly the best way for the project. A one-off sampling event at one monitoring well does not necessarily require the development of a custom digital field form. In this instance, a generic form that can be used by multiple projects may be suitable and more efficient than a project-specific form. Also, if your project site requires intrinsically safe equipment (that is, equipment incapable of releasing sufficient electrical energy to cause ignition of a hazardous atmospheric mixture), pen and paper might be the only option. In addition, standardized forms may not be appropriate for all types of field data collection, such as interviews, surveys, or other unique observations. Nevertheless, taking the time to design a well-crafted paper form for field team members to fill out, along with basic training, can make subsequent data management and quality control tasks go smoothly. 

Digital field data collection solutions can be planned and predesigned ahead of mobilization to the field. Currently available digital field forms can be customized to the task at hand. They can be simple, stand-alone forms or integrated with a larger EDMS. Digital field forms can be as simple as a single response per location or highly complex with automated equations and conditional statements, or anywhere in between. Most programs today allow for automated data upload to an EDMS when connectivity is available or restored. Digital field data collection reduces both data collection time and time required to analyze and distribute the data collected. As with the data being collected, if field conditions or scope of work change, digital field data collection forms can easily be updated and distributed to the field crew if they are in an area with cellular data or internet connectivity.

The decision to use paper or digital field forms has implications for field data quality as well. Considerations for the use of paper versus digital field forms related to the quality of field data are discussed in the Field Data Collection Quality Assurance and Quality Control (QA/QC) Fact Sheet.

Digital data can be stored, backed up, and secured, and allow for transparency and visibility to all stakeholders involved. As soon as field activities are complete, project teams get real-time access to the data, and data checks can begin immediately. At the conclusion of the project, data can be downloaded and submitted to the client or client representative. On the back end, every good field data collection plan needs proper data storage backup and security, as described in the Data Access, Sharing, and Security Fact Sheet.

With these considerations in mind, you can also use the Field Data Collection Decision Tree to aid you in your decision to use a paper or digital field data collection solution.

4 PAPER FIELD FORM DESIGN CONSIDERATIONS

In addition to the general field data form design considerations, the following paper form design considerations can result in more accurate written documentation and more streamlined field data collection, and can facilitate eventual transfer of paper forms to a digital format, if needed.

1. Ensure legible handwriting. The primary goal of paper form design is to ensure that the data recorded will be understandable to all future users, and legibility is of prime importance. Including simple user instructions at the top of a paper form can help improve handwriting quality. Instruct users to write plainly, carefully, and clearly. Inform users if the form is intended to be transferred to a digital format. Additional training considerations are provided in the Field Data Collection Training Best Practices Fact Sheet. If users are required to record field data using shorthand or specific reference values, provide those references to users either on the paper form itself, or as a separate durable resource such as a laminated sheet. Design a field form response with the knowledge of the expected data type and length of the database field it will call home. Do not allow users to record multiple lines of lengthy text if the ultimate database field is limited to 100 characters. 
2. Provide space to accommodate handwriting.  It is critical that a paper form can spatially accommodate handwriting. For portions of the form where typed text will be adjacent to handwritten field data, use an digital font size equivalent to an average handwriting size. Break up the form to allow space to accommodate handwriting. Where lengthy responses are anticipated, clearly identify where the user is intended to record their responses via larger boxes. Upon completion of the form design, print out a hard copy and practice populating the fields to determine whether the spaces provided are large enough to accommodate handwriting. 
3. Provide character length guidance. For data with known character length responses, consider providing square boxes for characters in place of blank lines and including any known special characters such as hyphens, slashes, or parentheses. Without these character restraints, users may run characters together or write in areas of the form that are not intended. 
4. Leverage lists and bubble fields. When field data are based on a selection from known reference values, provide users with lists and bubble fields, like those found on standardized tests, in place of blank lines. These forms are highly accurate and eliminate the need for handwritten responses. This can also facilitate the ultimate transfer of data to a digital format. Again, provide users with instructions on how to properly complete bubble fields, with a visual example of a completely filled bubble. Soil color and classification charts shown in Figure 2 below are examples of aids that can be used to ensure correct information is recorded on field forms.
   

   

5. Plan for digitization. If it is intended that the paper form will be transferred to a digital format, including a page border or registration corners and instructing users not to write outside those limits can help ensure that all recorded field data can be captured digitally. Including barcodes, QR codes, or other form identifiers on forms can also facilitate the digitization of paper forms. Ensure that these identifiers are in the same location on every page with sufficient white space surrounding them, for example, including an “office use only” area on the page. 
6. Plan to associate multimedia files. Even when recording field data on paper forms, there can be multimedia files that need to be associated with those paper forms (for example: photo, video, audio). Include a method for documenting and associating multimedia files on paper forms. Modern cameras record and store lots of data, including date, time, and even GPS location. Recording this information on paper forms can help later associate the multimedia files with the correct paper forms. However, also recording a text description of photo content such as the subject of the photo, where the photo was taken, what direction the photographer was facing, or even why the photo was taken can be immensely helpful.

5 DIGITAL FIELD FORM DESIGN CONSIDERATIONS

When moving to digital field form design, do not limit the format of your digital forms to match your original paper form. Digital forms are different from paper forms and should be treated so. Digital forms can be designed visually, have multiple pages that can hold additional information, and have a logical flow for entering data. Digital forms can be designed and redesigned for a specific project or task. The following are design considerations for digital forms.

1. Device selection and management. Determine the types of devices and software that will be used for digital data collection. Consideration should be given to ensuring that digital forms are compatible across different types of devices and different versions of software. Often software needs to be updated on a regular basis. Determine if the digital forms are to be used on individually owned and managed devices or shared devices. Additional information on hardware and software selection are included in the Other Considerations for Field Data Collection Fact Sheet.
2. Auto-completion/automated responses. When designing a digital form, consider the data being collected and make your form concise to the task at hand. Digital forms can include auto-complete fields to save time and avoid human error. Examples of automated responses include:
   • date/time of the reading or test
   • location data (pulled from a GPS enabled device)
   • name of the field technician collecting the information (if it’s connected to an application with log-in credentials)
3. Prepopulation. More complex digital field forms can have the ability to prepopulate complex field data within a digital field form if the digital form is connected to a data source. Digital field forms can have formulas or complex equations built into them to calculate values to display to the user as data are entered. This prepopulation and calculation can be combined to provide a field user with useful information while still in the field. One example would be prepopulating a digital field form with a groundwater well elevation, having the user enter a depth to groundwater for that location, and having the digital field form calculate the groundwater elevation and display it to the user.
4. Built-in help documentation. Digital form designs can program help documentation directly into the form fields. Help documentation can give guidance to the user about what is expected in that field answer. This help documentation can take several different forms, including:
   • on-screen documentation that is always displayed
   • warnings or errors that are shown only under specific conditions
   • pop-up messages to tell the user that needed fields have not been populated and must be completed to proceed
   • on-demand help that is displayed when a user selects to view help documentation
5. Reference value lists. Digital forms can be designed to only accept a response from a list of valid reference values, with default answers to a form field prepopulated with the most common expected answer so the user can make a selection and move on. For example, during drilling activities and soil sampling, the expected value is from a list of Unified Soil Classification System (USCS) soil criteria. For some types of questions, it can be difficult to anticipate all possibilities. In these instances, providing an “Other” or “Unknown” option and an additional text field where users can write in a response can be helpful. This provides users with guidance on what is preferred and flexibility to accommodate changing field conditions.
6. Conditional logic. Digital forms can be designed with conditional logic that will either expand on an answer given and require additional information or skip to the next section, saving time in the field. Form usability improves by showing only relevant questions. As stated above, text fields for users to write in other responses can be displayed exclusively when a user makes an “Other” selection. Digital forms can improve data quality with data validation upon collection if designed correctly. A field technician can avoid entering invalid information in a well-designed form, but that same form could restrict a user from saving the data if data are unavailable or invalid data are used in a required form field. Digital form designs should be tested by field crews prior to deployment; testing should be thorough and extensive using real data scenarios.
7. Data storage destination. Know where the form’s field data are going and if/how they will be shared. Design a field form response with the knowledge of the expected data type and length of the database field it will call home. Do not design a remark field in a digital form for 2,000 variable characters if the database field is limited to 100 characters. Alternatively, if you are collecting water level information and the result is the numeric measured depth to water, be sure the valid response is numeric and not an alphabetical character where the user could enter something other than a number (for example, DRY or NM for not measured). For numeric inputs, constrains can be applied to limit the maximum the number of significant figures or specify an exact number of required significant figures. For example, depth-to-water measurements are often to the hundredth, yet a numeric field will often automatically shorten trailing zeros.
8. Form user testing and training. Finally, test your form with one or more end users to ensure data flow. If possible, try it in a real-world scenario. Be sure that the data are being collected, but also that they are transmitted effectively. It is critical that your solution functions as expected, and user accepted testing is complete. Expect feedback from testing and make the changes or justify the design prior to deployment. See the Field Data Collection Training Best Practices Fact Sheet for additional information.

6 REFERENCES AND ACRONYMS

The references cited in this fact sheet, and the other ITRC EDM Best Practices fact sheets, are included in one combined list that is available on the ITRC web site. The combined acronyms list is also available on the ITRC web site.