Standard
Critical Assessment of Metagenome Interpretation : the second round of challenges. / Meyer, Fernando; Fritz, Adrian; Deng, Zhi Luo; Koslicki, David; Lesker, Till Robin; Gurevich, Alexey; Robertson, Gary; Alser, Mohammed; Antipov, Dmitry; Beghini, Francesco; Bertrand, Denis; Brito, Jaqueline J.; Brown, C. Titus; Buchmann, Jan; Buluç, Aydin; Chen, Bo; Chikhi, Rayan; Clausen, Philip T.L.C.; Cristian, Alexandru; Dabrowski, Piotr Wojciech; Darling, Aaron E.; Egan, Rob; Eskin, Eleazar; Georganas, Evangelos; Goltsman, Eugene; Gray, Melissa A.; Hansen, Lars Hestbjerg; Hofmeyr, Steven; Huang, Pingqin; Irber, Luiz; Jia, Huijue; Jørgensen, Tue Sparholt; Kieser, Silas D.; Klemetsen, Terje; Kola, Axel; Kolmogorov, Mikhail; Korobeynikov, Anton; Kwan, Jason; LaPierre, Nathan; Lemaitre, Claire; Li, Chenhao; Limasset, Antoine; Malcher-Miranda, Fabio; Mangul, Serghei; Marcelino, Vanessa R.; Marchet, Camille; Marijon, Pierre; Meleshko, Dmitry; Mende, Daniel R.; Milanese, Alessio; Nagarajan, Niranjan; Nissen, Jakob; Nurk, Sergey; Oliker, Leonid; Paoli, Lucas; Peterlongo, Pierre; Piro, Vitor C.; Porter, Jacob S.; Rasmussen, Simon; Rees, Evan R.; Reinert, Knut; Renard, Bernhard; Robertsen, Espen Mikal; Rosen, Gail L.; Ruscheweyh, Hans Joachim; Sarwal, Varuni; Segata, Nicola; Seiler, Enrico; Shi, Lizhen; Sun, Fengzhu; Sunagawa, Shinichi; Sørensen, Søren Johannes; Thomas, Ashleigh; Tong, Chengxuan; Trajkovski, Mirko; Tremblay, Julien; Uritskiy, Gherman; Vicedomini, Riccardo; Wang, Zhengyang; Wang, Ziye; Wang, Zhong; Warren, Andrew; Willassen, Nils Peder; Yelick, Katherine; You, Ronghui; Zeller, Georg; Zhao, Zhengqiao; Zhu, Shanfeng; Zhu, Jie; Garrido-Oter, Ruben; Gastmeier, Petra; Hacquard, Stephane; Häußler, Susanne; Khaledi, Ariane; Maechler, Friederike; Mesny, Fantin; Radutoiu, Simona; Schulze-Lefert, Paul; Smit, Nathiana; Strowig, Till; Bremges, Andreas; Sczyrba, Alexander; McHardy, Alice Carolyn.
In:
Nature Methods, Vol. 19, No. 4, 2022, p. 429-440.
Research output: Contribution to journal › Journal article › Research › peer-review
Harvard
Meyer, F, Fritz, A, Deng, ZL, Koslicki, D, Lesker, TR, Gurevich, A, Robertson, G, Alser, M, Antipov, D, Beghini, F, Bertrand, D, Brito, JJ, Brown, CT, Buchmann, J, Buluç, A, Chen, B, Chikhi, R, Clausen, PTLC, Cristian, A, Dabrowski, PW, Darling, AE, Egan, R, Eskin, E, Georganas, E, Goltsman, E, Gray, MA
, Hansen, LH, Hofmeyr, S, Huang, P, Irber, L, Jia, H, Jørgensen, TS, Kieser, SD, Klemetsen, T, Kola, A, Kolmogorov, M, Korobeynikov, A, Kwan, J, LaPierre, N, Lemaitre, C, Li, C, Limasset, A, Malcher-Miranda, F, Mangul, S, Marcelino, VR, Marchet, C, Marijon, P, Meleshko, D, Mende, DR, Milanese, A, Nagarajan, N
, Nissen, J, Nurk, S, Oliker, L, Paoli, L, Peterlongo, P, Piro, VC, Porter, JS
, Rasmussen, S, Rees, ER, Reinert, K, Renard, B, Robertsen, EM, Rosen, GL, Ruscheweyh, HJ, Sarwal, V, Segata, N, Seiler, E, Shi, L, Sun, F, Sunagawa, S
, Sørensen, SJ, Thomas, A, Tong, C, Trajkovski, M, Tremblay, J, Uritskiy, G, Vicedomini, R, Wang, Z, Wang, Z, Wang, Z, Warren, A, Willassen, NP, Yelick, K, You, R, Zeller, G, Zhao, Z, Zhu, S, Zhu, J, Garrido-Oter, R, Gastmeier, P, Hacquard, S, Häußler, S, Khaledi, A, Maechler, F, Mesny, F, Radutoiu, S, Schulze-Lefert, P, Smit, N, Strowig, T, Bremges, A, Sczyrba, A & McHardy, AC 2022, '
Critical Assessment of Metagenome Interpretation: the second round of challenges',
Nature Methods, vol. 19, no. 4, pp. 429-440.
https://doi.org/10.1038/s41592-022-01431-4APA
Meyer, F., Fritz, A., Deng, Z. L., Koslicki, D., Lesker, T. R., Gurevich, A., Robertson, G., Alser, M., Antipov, D., Beghini, F., Bertrand, D., Brito, J. J., Brown, C. T., Buchmann, J., Buluç, A., Chen, B., Chikhi, R., Clausen, P. T. L. C., Cristian, A., ... McHardy, A. C. (2022).
Critical Assessment of Metagenome Interpretation: the second round of challenges.
Nature Methods,
19(4), 429-440.
https://doi.org/10.1038/s41592-022-01431-4Vancouver
Meyer F, Fritz A, Deng ZL, Koslicki D, Lesker TR, Gurevich A et al.
Critical Assessment of Metagenome Interpretation: the second round of challenges.
Nature Methods. 2022;19(4):429-440.
https://doi.org/10.1038/s41592-022-01431-4Author
Meyer, Fernando ; Fritz, Adrian ; Deng, Zhi Luo ; Koslicki, David ; Lesker, Till Robin ; Gurevich, Alexey ; Robertson, Gary ; Alser, Mohammed ; Antipov, Dmitry ; Beghini, Francesco ; Bertrand, Denis ; Brito, Jaqueline J. ; Brown, C. Titus ; Buchmann, Jan ; Buluç, Aydin ; Chen, Bo ; Chikhi, Rayan ; Clausen, Philip T.L.C. ; Cristian, Alexandru ; Dabrowski, Piotr Wojciech ; Darling, Aaron E. ; Egan, Rob ; Eskin, Eleazar ; Georganas, Evangelos ; Goltsman, Eugene ; Gray, Melissa A. ; Hansen, Lars Hestbjerg ; Hofmeyr, Steven ; Huang, Pingqin ; Irber, Luiz ; Jia, Huijue ; Jørgensen, Tue Sparholt ; Kieser, Silas D. ; Klemetsen, Terje ; Kola, Axel ; Kolmogorov, Mikhail ; Korobeynikov, Anton ; Kwan, Jason ; LaPierre, Nathan ; Lemaitre, Claire ; Li, Chenhao ; Limasset, Antoine ; Malcher-Miranda, Fabio ; Mangul, Serghei ; Marcelino, Vanessa R. ; Marchet, Camille ; Marijon, Pierre ; Meleshko, Dmitry ; Mende, Daniel R. ; Milanese, Alessio ; Nagarajan, Niranjan ; Nissen, Jakob ; Nurk, Sergey ; Oliker, Leonid ; Paoli, Lucas ; Peterlongo, Pierre ; Piro, Vitor C. ; Porter, Jacob S. ; Rasmussen, Simon ; Rees, Evan R. ; Reinert, Knut ; Renard, Bernhard ; Robertsen, Espen Mikal ; Rosen, Gail L. ; Ruscheweyh, Hans Joachim ; Sarwal, Varuni ; Segata, Nicola ; Seiler, Enrico ; Shi, Lizhen ; Sun, Fengzhu ; Sunagawa, Shinichi ; Sørensen, Søren Johannes ; Thomas, Ashleigh ; Tong, Chengxuan ; Trajkovski, Mirko ; Tremblay, Julien ; Uritskiy, Gherman ; Vicedomini, Riccardo ; Wang, Zhengyang ; Wang, Ziye ; Wang, Zhong ; Warren, Andrew ; Willassen, Nils Peder ; Yelick, Katherine ; You, Ronghui ; Zeller, Georg ; Zhao, Zhengqiao ; Zhu, Shanfeng ; Zhu, Jie ; Garrido-Oter, Ruben ; Gastmeier, Petra ; Hacquard, Stephane ; Häußler, Susanne ; Khaledi, Ariane ; Maechler, Friederike ; Mesny, Fantin ; Radutoiu, Simona ; Schulze-Lefert, Paul ; Smit, Nathiana ; Strowig, Till ; Bremges, Andreas ; Sczyrba, Alexander ; McHardy, Alice Carolyn. / Critical Assessment of Metagenome Interpretation : the second round of challenges. In: Nature Methods. 2022 ; Vol. 19, No. 4. pp. 429-440.
Bibtex
@article{979b1528645048eda85a577b486d67d5,
title = "Critical Assessment of Metagenome Interpretation: the second round of challenges",
abstract = "Evaluating metagenomic software is key for optimizing metagenome interpretation and focus of the Initiative for the Critical Assessment of Metagenome Interpretation (CAMI). The CAMI II challenge engaged the community to assess methods on realistic and complex datasets with long- and short-read sequences, created computationally from around 1,700 new and known genomes, as well as 600 new plasmids and viruses. Here we analyze 5,002 results by 76 program versions. Substantial improvements were seen in assembly, some due to long-read data. Related strains still were challenging for assembly and genome recovery through binning, as was assembly quality for the latter. Profilers markedly matured, with taxon profilers and binners excelling at higher bacterial ranks, but underperforming for viruses and Archaea. Clinical pathogen detection results revealed a need to improve reproducibility. Runtime and memory usage analyses identified efficient programs, including top performers with other metrics. The results identify challenges and guide researchers in selecting methods for analyses.",
author = "Fernando Meyer and Adrian Fritz and Deng, {Zhi Luo} and David Koslicki and Lesker, {Till Robin} and Alexey Gurevich and Gary Robertson and Mohammed Alser and Dmitry Antipov and Francesco Beghini and Denis Bertrand and Brito, {Jaqueline J.} and Brown, {C. Titus} and Jan Buchmann and Aydin Bulu{\c c} and Bo Chen and Rayan Chikhi and Clausen, {Philip T.L.C.} and Alexandru Cristian and Dabrowski, {Piotr Wojciech} and Darling, {Aaron E.} and Rob Egan and Eleazar Eskin and Evangelos Georganas and Eugene Goltsman and Gray, {Melissa A.} and Hansen, {Lars Hestbjerg} and Steven Hofmeyr and Pingqin Huang and Luiz Irber and Huijue Jia and J{\o}rgensen, {Tue Sparholt} and Kieser, {Silas D.} and Terje Klemetsen and Axel Kola and Mikhail Kolmogorov and Anton Korobeynikov and Jason Kwan and Nathan LaPierre and Claire Lemaitre and Chenhao Li and Antoine Limasset and Fabio Malcher-Miranda and Serghei Mangul and Marcelino, {Vanessa R.} and Camille Marchet and Pierre Marijon and Dmitry Meleshko and Mende, {Daniel R.} and Alessio Milanese and Niranjan Nagarajan and Jakob Nissen and Sergey Nurk and Leonid Oliker and Lucas Paoli and Pierre Peterlongo and Piro, {Vitor C.} and Porter, {Jacob S.} and Simon Rasmussen and Rees, {Evan R.} and Knut Reinert and Bernhard Renard and Robertsen, {Espen Mikal} and Rosen, {Gail L.} and Ruscheweyh, {Hans Joachim} and Varuni Sarwal and Nicola Segata and Enrico Seiler and Lizhen Shi and Fengzhu Sun and Shinichi Sunagawa and S{\o}rensen, {S{\o}ren Johannes} and Ashleigh Thomas and Chengxuan Tong and Mirko Trajkovski and Julien Tremblay and Gherman Uritskiy and Riccardo Vicedomini and Zhengyang Wang and Ziye Wang and Zhong Wang and Andrew Warren and Willassen, {Nils Peder} and Katherine Yelick and Ronghui You and Georg Zeller and Zhengqiao Zhao and Shanfeng Zhu and Jie Zhu and Ruben Garrido-Oter and Petra Gastmeier and Stephane Hacquard and Susanne H{\"a}u{\ss}ler and Ariane Khaledi and Friederike Maechler and Fantin Mesny and Simona Radutoiu and Paul Schulze-Lefert and Nathiana Smit and Till Strowig and Andreas Bremges and Alexander Sczyrba and McHardy, {Alice Carolyn}",
note = "Publisher Copyright: {\textcopyright} 2022, The Author(s).",
year = "2022",
doi = "10.1038/s41592-022-01431-4",
language = "English",
volume = "19",
pages = "429--440",
journal = "Nature Methods",
issn = "1548-7091",
publisher = "nature publishing group",
number = "4",
}
RIS
TY - JOUR
T1 - Critical Assessment of Metagenome Interpretation
T2 - the second round of challenges
AU - Meyer, Fernando
AU - Fritz, Adrian
AU - Deng, Zhi Luo
AU - Koslicki, David
AU - Lesker, Till Robin
AU - Gurevich, Alexey
AU - Robertson, Gary
AU - Alser, Mohammed
AU - Antipov, Dmitry
AU - Beghini, Francesco
AU - Bertrand, Denis
AU - Brito, Jaqueline J.
AU - Brown, C. Titus
AU - Buchmann, Jan
AU - Buluç, Aydin
AU - Chen, Bo
AU - Chikhi, Rayan
AU - Clausen, Philip T.L.C.
AU - Cristian, Alexandru
AU - Dabrowski, Piotr Wojciech
AU - Darling, Aaron E.
AU - Egan, Rob
AU - Eskin, Eleazar
AU - Georganas, Evangelos
AU - Goltsman, Eugene
AU - Gray, Melissa A.
AU - Hansen, Lars Hestbjerg
AU - Hofmeyr, Steven
AU - Huang, Pingqin
AU - Irber, Luiz
AU - Jia, Huijue
AU - Jørgensen, Tue Sparholt
AU - Kieser, Silas D.
AU - Klemetsen, Terje
AU - Kola, Axel
AU - Kolmogorov, Mikhail
AU - Korobeynikov, Anton
AU - Kwan, Jason
AU - LaPierre, Nathan
AU - Lemaitre, Claire
AU - Li, Chenhao
AU - Limasset, Antoine
AU - Malcher-Miranda, Fabio
AU - Mangul, Serghei
AU - Marcelino, Vanessa R.
AU - Marchet, Camille
AU - Marijon, Pierre
AU - Meleshko, Dmitry
AU - Mende, Daniel R.
AU - Milanese, Alessio
AU - Nagarajan, Niranjan
AU - Nissen, Jakob
AU - Nurk, Sergey
AU - Oliker, Leonid
AU - Paoli, Lucas
AU - Peterlongo, Pierre
AU - Piro, Vitor C.
AU - Porter, Jacob S.
AU - Rasmussen, Simon
AU - Rees, Evan R.
AU - Reinert, Knut
AU - Renard, Bernhard
AU - Robertsen, Espen Mikal
AU - Rosen, Gail L.
AU - Ruscheweyh, Hans Joachim
AU - Sarwal, Varuni
AU - Segata, Nicola
AU - Seiler, Enrico
AU - Shi, Lizhen
AU - Sun, Fengzhu
AU - Sunagawa, Shinichi
AU - Sørensen, Søren Johannes
AU - Thomas, Ashleigh
AU - Tong, Chengxuan
AU - Trajkovski, Mirko
AU - Tremblay, Julien
AU - Uritskiy, Gherman
AU - Vicedomini, Riccardo
AU - Wang, Zhengyang
AU - Wang, Ziye
AU - Wang, Zhong
AU - Warren, Andrew
AU - Willassen, Nils Peder
AU - Yelick, Katherine
AU - You, Ronghui
AU - Zeller, Georg
AU - Zhao, Zhengqiao
AU - Zhu, Shanfeng
AU - Zhu, Jie
AU - Garrido-Oter, Ruben
AU - Gastmeier, Petra
AU - Hacquard, Stephane
AU - Häußler, Susanne
AU - Khaledi, Ariane
AU - Maechler, Friederike
AU - Mesny, Fantin
AU - Radutoiu, Simona
AU - Schulze-Lefert, Paul
AU - Smit, Nathiana
AU - Strowig, Till
AU - Bremges, Andreas
AU - Sczyrba, Alexander
AU - McHardy, Alice Carolyn
N1 - Publisher Copyright:
© 2022, The Author(s).
PY - 2022
Y1 - 2022
N2 - Evaluating metagenomic software is key for optimizing metagenome interpretation and focus of the Initiative for the Critical Assessment of Metagenome Interpretation (CAMI). The CAMI II challenge engaged the community to assess methods on realistic and complex datasets with long- and short-read sequences, created computationally from around 1,700 new and known genomes, as well as 600 new plasmids and viruses. Here we analyze 5,002 results by 76 program versions. Substantial improvements were seen in assembly, some due to long-read data. Related strains still were challenging for assembly and genome recovery through binning, as was assembly quality for the latter. Profilers markedly matured, with taxon profilers and binners excelling at higher bacterial ranks, but underperforming for viruses and Archaea. Clinical pathogen detection results revealed a need to improve reproducibility. Runtime and memory usage analyses identified efficient programs, including top performers with other metrics. The results identify challenges and guide researchers in selecting methods for analyses.
AB - Evaluating metagenomic software is key for optimizing metagenome interpretation and focus of the Initiative for the Critical Assessment of Metagenome Interpretation (CAMI). The CAMI II challenge engaged the community to assess methods on realistic and complex datasets with long- and short-read sequences, created computationally from around 1,700 new and known genomes, as well as 600 new plasmids and viruses. Here we analyze 5,002 results by 76 program versions. Substantial improvements were seen in assembly, some due to long-read data. Related strains still were challenging for assembly and genome recovery through binning, as was assembly quality for the latter. Profilers markedly matured, with taxon profilers and binners excelling at higher bacterial ranks, but underperforming for viruses and Archaea. Clinical pathogen detection results revealed a need to improve reproducibility. Runtime and memory usage analyses identified efficient programs, including top performers with other metrics. The results identify challenges and guide researchers in selecting methods for analyses.
U2 - 10.1038/s41592-022-01431-4
DO - 10.1038/s41592-022-01431-4
M3 - Journal article
C2 - 35396482
AN - SCOPUS:85127703163
VL - 19
SP - 429
EP - 440
JO - Nature Methods
JF - Nature Methods
SN - 1548-7091
IS - 4
ER -
