C1: Overview of data collection process

Statistical output quality is achieved through the statistical data collection process. Data development, i.e., the design, planning and organisation of statistical processes, is essential to achieve data quality. A statistical process will never maximise all of the quality components of a given statistical output at one time. For example, expert assessment might help to achieve the timeliness for the publication of statistical information at the expense of accuracy. Finally, end users will probably have a different perception of data quality than the data producers. Therefore, although user perception is dependent on the objective data quality achieved by the statistical processes, the users will often have subjective views and priorities in their assessing of the data quality. For example, among organic sector stakeholders, some believe that “inaccurate but timely data is better than no data”. User perception needs therefore to be considered when planning organic market data collection, as how the users actually perceive the quality of the statistical output is an essential part of any data quality assessment (Ehling and Koerner, 2007).

Data quality management starts from the development of survey instruments and questionnaires, and continues throughout the data production process itself, and then beyond, once the data are published. The costs of achieving data quality need to be balanced with the costs of producing data of poor quality (Table C.1-1).

Table C.1-1: A taxonomy of the data quality costs (Haug et al, 2011)

In the next sections, specific guidelines will be provided to apply Principles 7, 8, 9 and 10 of the Code of Practice, to achieve better quality (as defined in Principles 11-15) for the production of organic market data.

With respect to the data collection process, these issues will be addressed in the following sections:

Data collection methods need to be coherent with the type of data and the purpose of the data collection (section C2).

• Standard tools should be developed for the collection and analysis of the data, and these tools should be harmonised with existing European definitions, nomenclatures and classifications (section C3).
• Data analysis and processing need to be appropriate for the final use (i.e., publication, modelling), to make the statistical information relevant to the users. Data quality analysis is time consuming, but it helps to reduce errors in the data. Therefore, consistency checks that serve to screen the quality of the data are highly recommended to all organisations and individuals engaged in the collection and dissemination of organic market data. Data quality checks need to be prioritised according to the user needs, and whenever inconsistent data are found, better data should be sought. In addition, if only expert estimates are available, it is highly recommended to carry out cross-checks against other sources (section C4);
• Achieving higher data quality might impact on publication times. Accuracy needs to be balanced with timeliness and punctuality of data publication. If inconsistencies cannot be resolved when the data is being analysed, this should be indicated when disseminating the data. In addition, users might have expectations of open access to all kinds of organic market data. However, as the development, production and publication of good quality data are costly, data collectors need to identify the appropriate business model to provide adequate funding for the process (section C5);
• Sample size, sampling method, and coverage of the ‘universe’ have a strong impact on the resulting data quality, in terms of both relevance and accuracy. Although these issues are partially discussed in section C2, Part D further explores them.