An experimental approach to estimate uncertainty of diatom community analysis in the accreditation process

An experimental approach to estimate uncertainty of diatom community analysis in the accreditation process

Microchemical Journal 150 (2019) 104078 Contents lists available at ScienceDirect Microchemical Journal journal homepage:

481KB Sizes 1 Downloads 161 Views

Microchemical Journal 150 (2019) 104078

Contents lists available at ScienceDirect

Microchemical Journal journal homepage:

An experimental approach to estimate uncertainty of diatom community analysis in the accreditation process Camilla Puccinelli



, Stefania Marcheggiani1, Laura Mancini1

Department of Environmental and Health, Istituto Superiore di Sanità, Rome, Italy



Keywords: Diatom community Ecological methods Accreditation Quality assurance

Diatoms, belonging to the class Bacillariophyceae, are one of the biological elements required for ecological quality status (EQS) assessment according to the Water Frame Directive. Italy has adopted the Intercalibration Common Metric index (ICMi) to evaluate the EQS using diatoms. Developed through the InterCalibration exercise and validated at the European level, the ICMi still presents uncertainty elements related to ecological, spatial and temporal heterogeneities. Quality control assurance has become necessary for ecological tests due to their importance in data acquisition during monitoring activities. The aim of this work was to describe the approach used for the measurement of sampling and/or laboratory phase uncertainty associated with ICMI according to ISO/IEC 17025. Following ISO recommendations, we adopted the “top-down” approach to evaluate uncertainty from repeatability using experimental data. The study was designed to determine the variability of the method in relation to both the sampling and laboratory phases. The approach consisted of analyzing three samples, each with Bad, Moderate or High ecological quality status, ten times, plus ten replicates of sampling from the Moderate status site. Sampling, treatment, analysis, identification and counting were performed according to standard procedures. The ICMi was calculated for all samples. The experiment was performed under the following repeatability conditions: same operator, microscope and iconographic guides. We obtained a standard deviation of repeatability in each series. This study is an attempt to quantify, for the first time, the variability of the diatom analysis process through the value of the ICMi, trying to reduce at least the main sources of errors (identification and counting). It represents an approach to be followed for the accreditation process of a diatom-based laboratory test method according to ISO/IEC 17025.

1. Introduction Aquatic ecosystem protection and management were radically changed in 2000 with the adoption of the European Union Water Framework Directive (WFD) [1]. The assessment of the structure and functioning of surface water ecosystems is expressed as Ecological Quality Status (EQS). This latter is based on composition and abundances of biological communities (phytobenthos, aquatic invertebrates, macrophytes, fish fauna) analysis supported by physico-chemical and hydromorphological quality elements. For the first time, biological quality elements became the central elements and thus the basis of management decisions [2]. The ecological status classification for each waterbody is expressed as the Ecological Quality Ratio (EQR) that represents the deviation from

the least disturbed conditions (reference conditions), divided into five quality classes of increasing degradation, from High to Bad [1]. Diatoms, unicellular algae in the class Bacillariophyceae, were chosen as proxies for the entire phytobenthos for the ecological status assessment of freshwater in EU countries [3,4]. The use of diatoms as ecological indicators was recognized long before the WFD, thinking to the fact that the first diatom-based indices were developed in the early 1980s [5]. They are sensitive against physico-chemical parameters and inorganic and organic compounds and consequently they are used to investigate more specific impacts such as eutrophication and acidification [6–13]. Analysis procedures of each biological elements were updated by each European country to comply WFD requirements. In Italy, the sampling and sample treatment procedures was carried out by the

Corresponding author. E-mail address: [email protected] (C. Puccinelli). 1 All authors equally contributed to this work. ⁎ Received 15 February 2019; Received in revised form 7 June 2019; Accepted 8 July 2019 Available online 09 July 2019 0026-265X/ © 2019 Elsevier B.V. All rights reserved.

Microchemical Journal 150 (2019) 104078

C. Puccinelli, et al.

National Working Group [14,15], furthermore a harmonized exercise was done to identification and diatom counting through three interlaboratory comparisons between 2011 and 2016 [16–18]. In the contest of WG Ecological Status — Common Implementation Strategy (CIS) of WFD each Member State must have developed or updated the national method to EQS. In general, they consist of preexisting indices based on diatom communities that were updated or combined in order to be compliant with the WFD. Indeed, not only the composition or abundance of taxa is required but also the ratio of sensitive and tolerant species. The Italian national assessment method is the Intercalibration Common Metric index (ICMi) [19–21] and it consists of the Indice de Polluosensibilité Spécifique (IPS) and the Trophic Index (TI) [22,23] and is obtained by calculating the arithmetic average resulting from the EQR of these two indices. IPS and TI are based on Zelinka and Marvan's formula [24], which considers the weighted averages of taxa sensibility to pollution (i.e. nutrients, organic degradation), as well as pH and salinity. In particular, the IPS accounts for general quality estimates, while TI measures mainly nutrient load [25]. The Italy participated to InterCalibration (IC) exercises of the Mediterranean, Central Baltic and Alpine EcoRegions, performed to harmonize the classification status among European countries in order to ensure that good ecological status would correspond to the same level of ecological quality everywhere in Europe [26,27]. Furthermore, the national methods describe the complexity of biological community structures through a simple numerical value, which plays an important role in river basin management plans. Indeed, the monitoring results and ecological data from aquatic environments should be of known and verifiable quality, as the WFD requires [1]. Therefore, quality control assurance has become necessary for ecological tests. In this context, the objective of laboratory quality assurance is to verify the accuracy and precision of information obtained from analytical results and to ensure that data are suitable for decisionmaking; thus, a standard procedure of quality assurance must have and apply procedures to estimate the uncertainty of measurement [28]. Uncertainty evaluation should be performed by applying different methods, such as “bottom up” or “top down” approaches as indicate by standard methods [29,30]. In detail, the former requires the identification of each source of uncertainty and the quantification of each of them and theirs sum while, the top down approach is based on repeatability or reproducibility of the method [29,30]. In this work, we described the methodology chosen for accreditation of Intercalibration Common Metric index according to ISO/IEC 17025. This process has included the estimation of uncertainty starting from diatom sampling and ending with calculation of the index.

Table 1 Diatom samples and their replicates used for evaluation of repeatability. Diatom sample A B C D

Ecological status Bad Moderate High Moderate

Replicates analyzed 10 replicates per sample 10 replicates from the same sampling site

operation qualification because it is a valuable quality assurance tool for laboratory personnel. 2.1.3. Uncertainty measurement Uncertainty measurement of the ICMi was performed by estimating the repeatability (Table 1) and its standard deviation, using experimental data according to the indications of ISO/TS 21748 [30]. The diatom data set used in this study for the repeatability calculation consisted in 40 samples divided into the following series:

• Sample A was sampled at a Bad quality class river site and was analyzed 10 times • Sample B was sampled at a Moderate quality class river site and was analyzed 10 times • Sample C was sampled at a High quality class river site and was analyzed ten times • Sample D consisted in 10 replicates from the same site as Sample B; diatoms were collected ten times from different stones.

2.2. Diatom community analysis Sampling, treatment/analysis, identification and counting were performed according to the standard procedures CEN EN 13496 and CEN EN 14407 [31,32], while the ICMi was calculated as described in the official document on the method [20]. Diatoms were collected with a toothbrush from the surface from the surface of pebbles and stones, for a total sampled surface area of 100 cm2. In the laboratory, all samples were treated using hot hydrogen peroxide (30%) (Sigma-Aldrich, Milan, IT) and hydrochloric acid (37%) (Sigma-Aldrich, Milan, IT), and permanent slides were prepared using synthetic resin with high reflection index (Naphrax — Brunel Microscopes Ltd, Wiltshire, UK). Ten slides (replicates) were prepared from each of samples A, B and C, while one slide was prepared from each replicate of sample D, for a total of 40 slides. Diatom identification was based on the morphological analysis of frustules, from each slide was carried out at the species level using a light microscope with 100× magnification (MOTIC BA410E, Milan, IT) and image analysis software (HD 1080p lite, Milan, IT) to measure the morphological features of valves (the two halves of the cell wall) with the use of iconographic guides [33–38]. Important elements for taxonomic classification are the symmetry of the valve, its iso- or heteropolarity, the presence/absence and arrangement of the raphe, the arrangement of striae and punctae, the length and width, the number of striae in 10 μm. At least 400 valves were counted for each slide to provide quantitative data as required for application of the ICMi. The ecological status was assessed by calculation of the ICMi formula:

2. Material and methods 2.1. Methodological approach The methodological approach followed was based on ISO/IEC 17025 indications. This Standard specifies the general requirements for laboratory procedures in carrying out tests and/or calibrations, including sampling. It covers testing and calibration performed using standard methods, non-standard methods and laboratory-developed methods. The key factors to accreditation process of diatom assessment methods were describe below. 2.1.1. Instruments The microscope was calibrated using a certified micrometer slide, because measurements of diatoms (length, width and number of striae in 10 μm) are essential part in the identification process.

ICMi = (EQR_IPS + EQR_TI)/2 where EQR_IPS = Ecological quality ratio (observed value/reference value) of ìIPS and EQR_TI = Ecological quality ratio (observed value/ reference value) of ìTI.

2.1.2. Operator qualification The participation to Ring test exercises [16,17] was used to the 2

Microchemical Journal 150 (2019) 104078

C. Puccinelli, et al.

2.2.1. Repeatability condition of the test The identification and valve counting of the 40 slides were performed by the same operator using the same microscope and iconographic guides.

Table 3 Results of ICMi diatom analysis of the four series. Replicates

ICMi values of the diatom samples A

2.3. Data analysis

1 2 3 4 5 6 7 8 9 10

To asses distribution of ICMi data was used Shapiro-Wilk test [39] and to evaluate the Outlier data Huber test [40]. The meaningful of data with or without sampling was done with F test [41] using their variance according following formula:

Fobs > Ftab The variance (Fobs) is the ratio of maximum standard deviation (stot) and minimum standard deviation observed (slab) and Ftab is the value of F table distribution. The contribution to total variance of sampling activities was established by means of the following equation:

B 0.218 0.247 0.251 0.231 0.220 0.233 0.236 0.227 0.248 0.243

Mean Standard deviation (sr)

slab 2) = ssamp2 ;

ssamp = ( stot 2

1.079 0.953 0.972 1.021 1.098 1.008 0.989 0.987 0.974 0.951

0.606 0.504 0.521 0.544 0.586 0.591 0.520 0.602 0.561 0.555

Sample A

Sample B

Sample C

Sample D

0.235 0.012

0.550 0.017

1.003 0.050

0.559 0.037

slab 2) 3.2. Data analysis


•s •s •s

0.549 0.582 0.552 0.541 0.561 0.520 0.541 0.551 0.503 0.555


Table 4 Mean and standard deviation calculated for each series.

(ssamp2 + slab2) = stot 2 ; (stot 2


The Shapiro-Wilk test used to analyze the ICMi results of four series, due to small dataset (10 replicates), showed a normal distribution (p value = 0.05.). Only one ICMi value (replicate no. 9 of Sample B) was considered an outlier based on Huber test and it was excluded from the subsequent calculation. Furthermore, the ICMi mean value and repeatability standard deviation results were showed in table (Table 4). The variance was resulted of standard deviations obtained from the two samples with Moderate ecological status (Samples D and B) and was analyzed using the F test (p = 0.05) to assess if the variability including sampling was significantly greater than the variability without sampling. One of the relevant results of this study refers to the variance ratio between two samples (Fobs = 4.79) because it was significantly greater than the value of Ftab (FTab (df=9; df=8; p =0.05) = 3.39). Moreover, to estimate the contribution of sampling on total uncertainty was analyzed subtracting from total variability (stot) those obtained without sampling (slab) and the results was estimated to be ssamp = 0.033.

tot = standard

deviation obtained from data including sampling standard deviation obtained from data without sampling samp = standard deviation related to sampling lab =

3. Results 3.1. Diatom community structure and Intercalibration Common Metric index calculation ICMi values are obtained using the abundances of each diatom species, their sensitivity to nutrient coefficients and their reliability indicator values. Species identified only one time were not included in the calculation of the index. A list of diatom species was created for each series to investigate variability in terms of community composition. The highest number of species (39) was in sample D and the lowest in sample C (28). The mean number of species included in each ICMi calculation varied from a maximum of 26.9 species in Sample A to a minimum of 22.1 in sample B (Table 2). Sample A communities were dominated by Navicula veneta Kützing, Nitzschia capitellata Hustedt and Nitzschia linearis (Agardh) W Smith. Sample B and Sample D communities were characterized by abundances of Luticula goeppertiana (Bleisch) Mann, Luticula nivalis (Ehrenberg) Mann, Craticula halophila (Grunow ex Van Heurck) Mann and Surirella brebissonii var. kuetzingi Krammer Lange-Bertalot. Sample C was characterized by the dominance of Achnanthidium minutissimum (Kützing) Czarnecki, which ranged from 47% to 55% of the total count in the ten replicates. The results of the ICMi index analysis are reported in Table 3.

3.3. Uncertainty measurement assessment Expanded uncertainty (u) is established as:

u = 2*s where 2 is the coverage factor representing a 95% confidence interval. 4. Discussion and conclusion Quality control and quality assurance actions should ensure the completeness and correctness of the data processing according to standard procedures [28]. It is long time that chemical and microbiological analyses are performed according to ISO/IEC 17025 while only in recently years become important for ecological methods too. The WFD already requires that chemical analyses be performed according to ISO/IEC 17025 on the contrary there aren't indications referring to ecological methods. Poor or lack information on quality assurance and quality control approaches used in biomonitoring are available in the scientific literature [42]. Only in one recent review [43] has mentioned the quality assurance aspects for diatom taxonomic identification. The conclusion of this work highlighted how ring tests results should be well

Table 2 Number of diatom species found in the four series and involved in ICMi calculation. Samples





Number of species identified in each series Mean number of species involved in ICMi calculation in each slide Maximum number of species involved in ICMi calculation in each slide Minimum number of species involved in ICMi calculation in each slide

29 26.9

35 22.1

28 22.2

39 22.7










Microchemical Journal 150 (2019) 104078

C. Puccinelli, et al.

documented and widespread to improve and consolidated the quality assurance approach in ecological field. For this reason, we tried to adapt the accreditation requirements and methods already provided for chemical and microbiological tests. The procedure described in this work represents an empirical approach to adapt the recommendations of ISO/IEC 17025 to ecological methods, including the uncertainty measurement analysis. Ecological methods, including the ICMi, have particular uncertainty elements that can be related to environmental factors such as temperature, light, nature of the substratum, nutrient availability, space, natural successional processes, grazing and hydromorphological regime [44,45]. Furthermore, they have uncertainty due to analytical factors [46–49]. It was not possible to achieve a reasonable estimation of each of these components, assuming that all of them are completely independent. For this reason, we assessed expanded uncertainty by adopting the “top-down” approach based on performance data of the whole method, as also used to calculate uncertainty for ecotoxicological assays and microbiological tests [50,51]. There was no indication of reproducibility data of the ICMi in the European Standard [32] to be compared with our results, and we took into consideration the repeatability of our experimental data also because it was required to be used in internal quality controls. Several works such as the European ring test and other harmonization exercises have indicated that identification and counting are important sources of uncertainty in the analysis of diatoms, although the critical phase is still the identification process, especially when many analysts are involved [15,42,45,46]. Performing the diatom test in conditions of repeatability (same operator, microscope and iconographic guides), reduced to the minimum the uncertainty due to identification and also due to counting, and allowed quantification of the variability due to the laboratory phase, including sample treatment and slide preparation. In terms of number of species, the maximum variability was found in the Moderate ecological status sample; this was due to greater assemblage heterogeneity where tolerant and sensitive taxa can co-exist [44]. For Bad and Moderate status (Sample A and Sample B), the ICMi method revealed similar variation of the relative abundances of the most representative species; indeed, the standard deviation values were 0.012 and 0.017 (Table 4). A higher standard deviation (0.050) was found for the High ecological status (Sample C), probably due to the dominance of one species, Achnanthidium minutissimum, in this diatom community. Further studies and also ring tests should be performed to better understand this variability in the index calculation, and more precise and restrictive counting protocols should be prepared. Sample D (10 replicates from the same river site) presented a higher number of identified species, but a mean of 22.7 species included in the ICMi calculation, a value similar to those of Samples B and C (Table 2); this was probably due to laboratory variability. Sampling was performed by collecting the replicates of diatoms pebbles and stones, on the same day, at the same river sites. Indeed, the main sources of sampling uncertainty depend on several aspects such as sampling site choice, choice of substrates and sample collecting [46,47,49,52]. When the sampling protocol is not strictly respected, this part of the variability can be very high [53], even if it is correctly applied, sampling activities still represent a main source of total uncertainty, considering the F test results and the standard deviation value (0.033). It could mean that sampling activities can strongly influence the total variability not also for moderate state but reasonable for other statuses. To date, only a few biological assessment methods have estimated their uncertainty. Including this estimation in assessment schemes is a major challenge for the next phase of WFD implementation [2]. The standardization of analytical protocols could decrease the uncertainty resulting from the procedures in order to make the data reproducible and comparable. The uncertainty assessed in this study is that related mainly to variation in sample preparation or to environmental variability from

different natural hard substrates. We have attempted to quantify, for the first time, the variability of the diatom analysis process through the value of the ICMi, trying to reduce at least the above-mentioned main sources of errors. Furthermore, we have presented an approach to be followed for the accreditation process of a diatom-based laboratory test method according to ISO/IEC 17025. However, for better reliability of surface water monitoring using diatom-based methods, stricter protocols for counting should be devised and more training courses to harmonize identification should be carried out at the national and, better yet, European level [42]. In conclusion, the experimental approach adopted in this study represents another important improvement in the development of biotic diatom-based indices, which will significantly enhance their use in surface water monitoring. Acknowledgements Thanks to Federico Pecoraro for technical support during the accreditation process of our laboratory, and thanks to Fabrizio Volpi for support in the accreditation of the diatom test. Declaration of Competing Interest The authors declare no conflict of interest. References [1] CEC - Council of European Communities, Directive 2000/60/EEC of 23 October 2000 establishing a framework for community action in the field of water policy. Luxembourg, Off. J. Eur. Communities 327 (2000) (22.12.2000). [2] D. Hering, A. Borja, J. Carstensen, L. Carvalho, M. Elliott, C.K. Feld, W. van de Bund, The European water framework directive at the age of 10: a critical review of the achievements with recommendations for the future, Sci. Total Environ. 408 (2010) 4007–4019, [3] W. Van de Bund, Water Framework Directive Intercalibration Technical Report, Office for Official Publications of the European Communities, European Commission Luxembourg, 2009 (2009. Available on line at Last access 14/02/2019.). [4] M.G. Kelly, S. Juggins, R. Guthrie, S. Pritchard, J. Jamieson, B. Rippey, H. Hirst, M. Yallop, Assessment of ecological status in UK rivers using diatoms, Freshw. Biol. 53 (2008) 403–422. [5] F. Rimet, Recent views on river pollution and diatoms, Hydrobiologia 683 (2012) 1–24. [6] M.G. Kelly, B.A. Whitton, The Trophic Diatom Index: a new index for monitoring eutrophication in rivers, J. Appl. Phycol. 7 (1995) 433–444. [7] A. Dell’Uomo, L’Indice Diatomico di Eutrofizzazione/Polluzione (EPI-D) nel monitoraggio delle acque correnti, Linee Guida APAT, CTN AIM, Roma, 2004. [8] G. Várbíró, G. Borics, B. Csányi, G. Fehér, I. Grigorszky, K.T. Kiss, A. Tóth, É. Ács, Improvement of the ecological water qualification system of rivers based on first results of the Hungarian phytobenthos surveillance monitoring, Hydrobiologia 695 (2012) 125–135. [9] S. Blanco, L. Ector, E. Bécares, Epiphytic diatoms as water quality indicators in Spanish shallow lakes, Vie Milieu 54 (2004) 71–79. [10] D.M. DeNicola, E. Eyto, Using epilithic algal communities to assess trophic status in Irish lakes, J. Phycol. 40 (2004) 481–495. [11] B. Bolla, G. Borics, K.T. Kiss, Reskóné, M. Nagy, G. Várbíró, É. Ács, Recommendations for ecological status assessment of Lake Balaton (largest shallow lake of Central Europe), based on benthic diatom communities, Vie Milieu—Life Environ. 60 (2010) 197–208. [12] J. Prygel, M. Coste, Les diatomées et le diagnostic de la qualité des eaux courants continentales: les principales méthodes indicielles, Vie Milieu 45 (1995) 179–186. [13] T. Kauppila, T. Moisio, V.P. Salonen, A diatom-based inference model for autumn epilimnetic total phosphorus concentration and its application to a presently eutrophic boreal lake, J. Paleolimnol. 27 (2002) 261–273. [14] ISPRA, Metodi biologici per le acque. Parte I, ISPRA, Roma, 2007 Disponibile all’indirizzo acque.html; , Accessed date: 21 May 2009. [15] ISPRA, Metodi biologici per le acque superficiali interne. Seduta del 27 novembre 2013 - Doc. n. 38/13CF, Istituto Superiore per la Protezione e la Ricerca Ambientale, Roma, 2014 (Manuali e Linee Guida 111/2014). [16] C. Martone, C. Vendetti, C. Puccinelli, S. Balzamo, S. Barbizzi, S. Marcheggiani, G. Benedettini, L. Mancini, Data quality in ecological status assessment on diatom communities, Res. J. Life Sci. Bioinf. Pharma. Chem. Sci. 3 (3) (2017) 194. [17] AAVV, Prova valutativa interlaboratorio Ispra-ic030 “Identificazione tassonomica delle diatomee bentoniche – Metodo ICMi” Rapporto Conclusivo, Istituto Superiore per la Protezione e la Ricerca Ambientale, Roma, 2015.


Microchemical Journal 150 (2019) 104078

C. Puccinelli, et al.

[35] K. Krammer, H. Lange-Bertalot, Bacillariophyceae 3 Teil: Centrales, Fragilariaceae, Eunotiaceae, in: H. Ettl (Ed.), Suβwasserflora von, Gustav Fischer-Verlag, Mitteleuropea Stuttgart, 1991. [36] K. Krammer, H. Lange-Bertalot, Bacillariophyceae 4 Teil: Achnathaceae Kritische Erganzungen zu Navicula und Gomphonema, in: H. Ettl (Ed.), Suβwasserflora von Mitteleuropea, Gustav Fischer-Verlag, Stuttgart, 1991. [37] K. Krammer, H. Lange-Bertalot, Bacillariophyceae 5 Teil: English and French translation of the keys, in: H. Ettl (Ed.), Suβwasserflora von, Gustav Fischer-Verlag, Mitteleuropea Stuttgart, 2000. [38] . Hofmann, G., Werum, M., Lange-Bertalot, H. 2013. Diatomeen im SüßwasserBenthos von Mitteleuropa. Koeltz Scientific Books, 61462 Königstein, 908 pp. Krammer & Lange-Bertalot, 1986; (2000); [39] S.S. Shapiro, M.B. Wilk, An analysis of variance test for normality (complete samples), Biometrika 52 (1965) 591–611. [40] P.J. Huber, Robust Statistics, John Wiley & Sons, Inc., New York, 1981 (MR 0606374). [41] G.E.P. Box, Non-normality and tests on variances, Biometrika 40 (1953) 318–335 JSTOR 2333350 [42] I. Lavoie, S. Campeau, Assemblage diversity, cell density and within-slide variability: implications for quality assurance/quality control and uncertainty assessment in diatom-based monitoring, Ecol. Indic. 69 (2016) 415–421. [43] M. Kahlert, É. Ács, S.F.P. Almeida, S. Blanco, M. Dreßler, L. Ector, S.M. Karjalainen, A. Liess, A. Mertens, J. van der Wal, S. Vilbaste, P. Werner, Quality assurance of diatom counts in Europe: towards harmonized datasets, Hydrobiologia 772 (2016) 1–14. [44] M. Kelly, H. Bennion, A. Burgess, J. Ellis, S. Juggins, R. Guthrie, J. Jamieson, V. Adriaenssens, M. Yallop, Uncertainty in ecological status assessments of lakes and rivers using diatoms, Hydrobiologia 633 (2009) 5–15. [45] S.I. Passy, Spatial paradigms of lotic diatom distribution: a landscape ecology perspective, J. Phycol. 37 (2001) 370–378. [46] M. Lindegarth, J. Carstensen, R.K. Johnson, Uncertainty of Biological Indicators for the WFD in Swedish Water Bodies: Current Procedures and a Proposed Framework for the Future, Havsmiljöinstitutet, Sweden, 2013 (Deliverable 2.2-1, WATERS Report no. 2013:1). [47] M. Kahlert, M. Kelly, R.-L. Albert, S.F.P. Almeida, T. Bešta, S. Blanco, Identification is a minor source of uncertainty in diatom-based ecological status assessments on a continent-wide scale: results of a European ring-test, Hydrobiologia 695 (2012) 109–124. [48] R.T. Clarke, D. Hering, Errors and uncertainty in bioassessment methods—major results and conclusions from the STAR project and their application using STARBUGS, Hydrobiologia 566 (2006) 433–439. [49] J. Carstensen, Statistical principles for ecological status classification of Water Framework Directive monitoring data, Mar. Pollut. Bull. 55 (2007) 3–15. [50] C. Ledda, G. Oliveri Conti, M. Fiore, R. Fallico, S. Sciacca, M. Ferrante, Measurement uncertainty for the determination of the inhibitory effect on Vibrio fischeri: a practical approach, J. Environ. Chem. Ecotoxicol. 3 (2011) 139–141. [51] B. Jarvis, J.E. Corry, A.J. Hedges, Estimates of measurement uncertainty from proficiency testing schemes, internal laboratory quality monitoring and during routine enforcement examination of foods, J. Appl. Microbiol. 103 (2007) 462–467. [52] A.P. Besse-Lototskaya, F.M. Verdonschot, J.A. Sinkeldam, Uncertainty in diatom assessment: sampling, identification and counting variation, Hydrobiologia 566 (2006) 247–260. [53] J. Prygiel, P. Carpentier, S. Almeida, M. Coste, L. Ector, D. Guillard, M.A. Honore, R. Iserentant, P. Ledeganck, C. Lalanne-Cassou, C. Lesniak, I. Mercier, P. Moncaut, M. Nazart, N. Nouchet, P.V. Peres, F. Rimet, A. Rumeau, S. Sabater, F. Straub, M.C. Torrisi, L. Tudesque, B. Van De Vijver, H. Vidal, J. Vizinet, N. Zydek, Determination of the biological diatom index (IBD NF T 90–354): results of an intercomparison exercise, J. Appl. Phycol. 14 (2002) 27–39.

[18] M. Torrisi, C. Monauni, R. Zorza, V. Della Bella, M. Siligardi, C.E. Wetzel, L. Ector, Ring-test su identificazione e protocollo di conteggio di diatomee bentoniche svolto all’Agenzia Provinciale per la Protezione dell’Ambiente di Trento (APPA), Biol. Ambient. 28 (2014) 113–127. [19] M. Kelly, C. Bennett, M. Coste, F. Delmas, L. Denys, L. Ector, C. Fauville, M. Ferreol, M. Golub, M. Kahlert, J. Lucey, B. Ni Chathain, I. Pardo, P. Pfister, J. PicinskaFaltynowicz, C. Schranz, J. Schaumburg, J. Tison, H. van Dam, S. Vilbaste, Central/ Baltic GIG Phytobenthos Intercalibration Exercise — Final Report, Bowburn Consultancy, Durham, 2006. [20] L. Mancini, C. Sollazzo (Eds.), Metodo per La Valutazione Dello Stato Ecologico Delle Acque Correnti: Comunità Diatomiche, Istituto Superiore di Sanità, Roma, 2009(Rapporti ISTISAN 09/19). [21] Italy, Ministerial Decree (DM) 260/2010 of Nov. 2010. Regolamento recante «Criteri tecnici per la classificazione dello stato dei corpi idrici superficiali, per la modifica delle norme tecniche del decreto legislativo 3 aprile 2006, n. 152, recante norme in materia ambientale, predisposto ai sensi dell'articolo 75, comma 3, del medesimo decreto legislativo». Official Journal no. 31, 7 February 2011, (2010). [22] CEMAGREF, Étude des méthodes biologiques d’appreciation quantitative de la qualité des eaux, Rapport Q.E. Lyon - A.F. Bassin Rhône-Mediterranée Corse. Lyon, 1982. [23] E. Rott, P. Pfister, H. van Dam, E. Pipp, K. Pall, N. Binder, K. Ortler, Indikationslisten für Aufwuchsalgen in Österreichischen Fliessgewässern, Teil 2: Trophieindikation und autökologische Anmerkungen Bundesministerium für Landund Forstwirtschaf, Wasserwirtschaftskataster, Wien, 1999. [24] M. Zelinka, P. Marvan, Zur Prazisierung der biologischen Klassifikation der Reinheit fliessender Gewasser, Arch. Hydrobiol. 57 (1961) 398–407. [25] S.F.P. Almeida, C. Elias, J. Ferreira, E. Tornés, C. Puccinelli, F. Delmas, G. Dörflinger, G. Urbanič, S. Marcheggiani, J. Rosebery, L. Mancini, S. Sabater, Water quality assessment of rivers using diatom metrics across Mediterranean Europe: a methods intercalibration exercise, Sci. Total Environ. 476–477 (2014) 768–776. [26] European Commission, Common Implementation Strategy (CIS) Guidance Document No 14, Guidance on the Intercalibration Process 2004–2006, Brussels (2005). [27] European Commission, COMMISSION DECISION of 20 September 2013 Establishing, Pursuant to Directive 2000/60/EC of the European Parliament and of the Council, the Values of the Member State Monitoring System Classifications as a Result of the Intercalibration Exercise and Repealing Decision 2008/915/EC, (2013). [28] ISO/IEC 17025, General Requirements for the Competence of Testing and Calibration Laboratories, International Organization for Standardization, Geneva, 2005. [29] UNI CEI 70098-3, Incertezza di misura - Parte 3: Guida all'espressione dell'incertezza di misura, (2016). [30] ISO/TS 21748, Guidance for the Use of Repeatability, Reproducibility and Trueness Estimates in Measurement Uncertainty Estimation, International Organization for Standardization, Geneva, 2004. [31] CEN EN 13946, Water Quality — Guidance for the Routine Sampling and Preparation of Benthic Diatoms from Rivers and Lakes, European Committee for Standardization, Brussels, 2014. [32] CEN EN 14407, Water Quality — Guidance for the Identification and Enumeration of Benthic Diatom Samples from Rivers and Lakes, European Committee for Standardization, Brussels, 2014. [33] K. Krammer, H. Lange-Bertalot, Bacillariophyceae 1 Teil: Naviculaceae, in: H. Ettl (Ed.), Suβwasserflora von Mitteleuropea Stuttgart, Gustav Fischer-Verlag, 1986. [34] K. Krammer, H. Lange-Bertalot, Bacillariophyceae 2 Teil: Bacillariaceae, Epithemiaceae, Surirellaceae, in: H. Ettl (Ed.), Suβwasserflora von Mitteleuropea, Gustav Fischer-Verlag, Stuttgart, 1988.