Step 3: Organize Your Data and Ensure Data Quality

Good kitchen organization is necessary to ensure food is prepared safely and properly. Similarly, organizing data and ensuring data quality are necessary precursors to data analysis.

---

Organize data

To organize your quantitative data, there are a wide range of database management systems – relational, network, flat, and hierarchical – that can be customized in order to perform the following tasks relevant to your evaluation data:

• Defining data – Creating data definitions and relationships to organize the data (e.g. participant ID link to survey responses, street segment ID link to items from an environmental audit, city ID link to policy assessment data, county ID link to sociodemographic variables, and latitude and longitude coordinates link to spatial data for mapping) as well as setting parameters for valid values for each variable.
• Entering data – Designing forms for inputting new data, modifying existing data, or deleting unwanted data.
• Downloading or reporting data – Developing usable formats for transferring data for further processing in data analysis applications or sharing with different audiences.
• Administration – Registering and monitoring users, enforcing data security and privacy, monitoring system performance, maintaining data integrity, and recovering information corrupted by an event.

To organize and properly name and store your qualitative data records (e.g., audio files, transcripts, consent forms, observation notes/rubrics), there are a few essential components, including:

• Identifying metadata to archive records, including data collection methods (e.g., interviews, focus groups, observations), dates or times of data collection, names of those responsible for data collection, individuals or groups of participants, and names of those responsible for coding or reviewing the analysis.
• Centralizing and organizing records in a location with proper filenames based on metadata (e.g., "[Individual or Group Name]_[Method Type]_[Date]_[Time]") as well as cataloging or grouping the records and documents in folders by method type.
• Creating a data tracking system to identify files ready for transcription or analysis as well as storing original records archives separate from versions used for analysis.

---

B. Ensure data quality

Several steps can be taken to ensure data quality during data collection (Section 4), such as using tools with established validity and reliability and engaging multiple people in data collection to establish inter-observer reliability.

Likewise, good data analysis involves several complementary steps to ensure data quality prior to analysis, including: using multiple data analysts, creating a data analysis protocol, training data analysts, reviewing data validity and reliability, addressing missing information, and addressing bias.

Using multiple coders or data analysts is particularly helpful for qualitative data analysis to reconcile differences or subjective interpretations of the data. While multiple coders or analysts require more organization, time, and money, the resulting themes or codes are typically higher quality, given the opportunity to discuss varying interpretations and generate consensus on the most appropriate themes or codes.

Your efforts to maximize data quality help to minimize threats to internal and external validity, and can increase confidence in the findings from the data analysis (causal inference).

You and your partners should discuss the following data quality control procedures to ensure that the evaluation findings demonstrate changes in the outcomes of interest attributed to the intervention as opposed to alternative explanations.

View the Resource Table

---

Next: Continue to Step 4