This website was archived on January 11, 2020 and is no longer updated.
D.8 Step 8: Quality checks
Quality checks should be done continuously, starting from the definition of your aims until data analysis. In the description of the individual steps, relevant quality issues have been mentioned. At this point, we will focus on the quality checks after you have finished data collection.
How to check the plausibility of your organic market data
After you have finished collecting your data, but before you start doing any calculations, analyses, or diagrams, you have to check your data for any inconsistencies, implausible values, or errors. It is necessary to trace back all inconsistencies: First, in order to correct the figures in your current dataset (the file which includes the data you collected), and second, to avoid the same mistakes in future surveys. If you cannot find the reasons for certain inconsistencies, you have to delete the data from your database. The list below serves as a guideline on how to carry out plausibility checks.
- Compare results for certain parameters (e.g. organic area, production, sales) between different years (e.g. 2010 and 2011)
You can easily identify inconsistencies in your data by comparing your data with a dataset from the previous year, which is from a source that includes the same values. You should look for extreme increases or decreases in values: Changes of more than 25% should attract your attention and you should check your data again. However, in some cases even great changes may reflect real developments, e.g. production volumes can be very different due to extreme weather conditions. - Compare yields between countries with similar farming conditions
This is another fairly simple way of checking the plausibility of your data. However, it should never be applied as the only data plausibility check, but only in combination with other checks. As prerequisites, you need to have access to comparable data from other countries and a good knowledge about the farming conditions, production characteristics and other parameters that influence agricultural yields. To ensure that yield data from other countries is comparable to data from your country, you first have to check whether the total (i.e. conventional) yields for these countries are similar. In this case you can use them for further comparisons. - Check: Organic production share < organic area share
The key aspect of this plausibility check is that the share of organic production from total production (e.g. of a certain crop) should be smaller than the share of organic production area from total agricultural area. First, you have to carry out a few basic calculations: You divide the organic area (in ha) by the total (including conventional) area (in ha). You can get the necessary information from Eurostat or national statistics. You do the same calculation with organic production (e.g. of wheat, usually specified in tonnes) and total production.
If your results show that the share of organic production is higher than the total share of organic area, it is likely that there is a mistake in your set of organic data, as yields are normally lower in organic agriculture than in conventional agriculture.
- Check: Organic yield < conventional/total yield
This plausibility check is closely related to the previous, as in most cases organic yields are smaller than total or conventional yields. When your results are different, you need to check whether there is a natural reason for this occurrence or whether there is a mistake in your data. - Check: Organic area < total area
Again, this is similar to the previous checks: Organic areas must be smaller than total areas which are reported in official statistics. Other results suggest an inconsistency in the data. Errors can have occurred in data handling, e.g. if you have used different units (e.g. 1000 ha instead of million ha) or through typing mistakes in data entry (decimal point in the wrong place). - Check: Imports ≤ sales
This check for inconsistencies is only applicable for those countries that collect organic trade data. Imports for a certain product should be smaller than sales for the same product in the same country. In case imports are higher than sales of a certain product, there either must be a mistake in the respective data or there must be significant exports at the same time. - Check: Domestic organic consumption = organic sales (products sold as organic + organic imports – organic exports)
This check for inconsistencies is only applicable for those countries that collect organic trade data. It might to be difficult to get the necessary data to carry out this check, but the results can reveal interesting insights into the functioning of the organic market in your country. A way to get organic sales and import data is through wholesalers. During data collection, you should differentiate between products used for human consumption and products used as animal feed or for other purposes (e.g. energy production).
The checklist helps you to get a short overview over the different ways of checking your data for plausibility:
Checklist: Are your results plausible?
- Are there extreme changes in one parameter (e.g. organic production) between two years? If yes: Is there an explanation for these changes?
- Are the results for yields similar to those from comparable countries?
- Is the organic production share smaller than the organic area share?
- Is the organic yield smaller than the conventional or total yield?
- Is the organic area smaller than the total area?
- Are the organic imports smaller than the organic sales?
- Is the domestic organic consumption identical to the organic sales?
