0% READ
FULL TEXT
INTRODUCTION
Pathway analysis has become an indispensable step in the interpretation of high-dimensional molecular data, enabling researchers to move from lists of differentially expressed genes/proteins toward a higher-level understanding of biological mechanisms. By identifying perturbed signaling or metabolic pathways, researchers can uncover insights into disease processes, drug responses, and cellular functions, facilitating the interpretation of omics data.
The advent of multi-omics studies, which integrate data from multiple molecular layers has stimulated the need for sophisticated pathway analysis tools capable of handling heterogeneous datasets. In response to this demand, the research community has developed a plethora of pathway analysis tools tailored for multi-omics integration. Prominent examples include ClusterProfiler,[1] PARADIGM,[2] Pathview,[3] PaintOmics,[4] and ActivePathways.[5] These tools vary in terms of supported data types, statistical approaches, visualization capabilities, and usability.
Several recent reviews have evaluated multi-omics integration tools from methodological perspectives. Maghsoudi et al.[6] provided a comprehensive overview of pathway-analysis approaches, highlighting their assumptions, supported data types, and practical limitations. A classification of integration methods according to translational-medicine objectives was proposed by Athieniti and Spyrou.[7] Morabito et al.[8] provided a summary of algorithmic classes for data-driven omics integration, while network-based frameworks linking multi-omics data to biomarker and interaction discovery were reviewed by Kumar and colleagues.[9]
However, while many of the multi-omics pathway tools are widely cited in the scientific literature, their actual adoption in applied research studies remains unclear. For example, some tools may accumulate citations because of theoretical significance or pioneering contributions to the field, yet their actual integration into research pipelines could be limited. This raises a question: which pathway analysis tools are actively utilized in modern multi-omics research?
To address this gap, we conducted a systematic evaluation of the practical usage of multi-omics pathway analysis tools. Rather than aiming to compile the most up-to-date list of tools, we focused on a set of methods that are already established in the field and have had sufficient time to accumulate citations and be evaluated in practical applications. For this reason, we based our analysis on the well-curated overview[6] of multi-omics pathway analysis tools. Using this reference set, we examined citations in open-access research papers, distinguishing between citations in the Methods section (indicating actual tool employment) and those in the Introduction or Discussion sections, suggesting use in literature reviews. Our goal was to identify the "working horses" of multi-omics pathway analysis and provide a clearer picture of their practical impact. To our knowledge, this is the first quantitative analysis of the gap between citation popularity and actual use of pathway analysis tools in multi-omics studies.
METHODOLOGY
We compiled a list of pathway analysis tools designed for or adaptable to multi-omics data. Our goal was to select tools that were published not earlier than three years ago. We based our study on a comprehensive survey[6] of 32 tools, but removed tools that were not freely available to the public since limited accessibility could introduce bias into the assessment of real-world usage. We chose to retain PARADIGM[2] tool in the analysis, which is not currently publicly available, but was accessible in the past.
For each tool, we searched for open-access papers that cited the tool’s primary publication(s), excluding paywalled articles to ensure the reproducibility of our analysis. We used PubMed to identify relevant publications, focusing on papers published before 2025 (the citation collection was carried out in March-April 2025).
The search was restricted to English-language papers. When a tool was described in multiple publications, we included all relevant ones. For example, for Mergeomics software, we used both the original paper[10] describing version 1.0 and the following paper[11] describing version 2.0. At the same time, we ensured that the selected methods were explicitly designed for multi-omics analysis, e.g. in the case of GeneTrail, we excluded the earliest publication[12] because it described a single-omics version of the tool.
For each publication, we retrieved the corresponding full-text XML file in NISO-JATS format from the PubMed Central database (“ANSI/NISO Z39.96-2024, JATS: Journal Article Tag Suite, version 1.4”). Papers for which the publisher did not permit full-text XML download were excluded from the analysis. The downloaded XML files were processed using the JATSdecoder package, which extracts structured metadata, abstracts, sectioned body text, and reference lists from JATS-formatted articles.[13]
Paper was considered a research article if the `article-type` tag was equal to `Research Article`. For research articles, we determined whether the tool was actually used in the study by identifying citations that appeared within methodological sections of the text. This was done by scanning section titles using the regular expression ‘method procedur protocol’. Citations found within these sections were considered indicative of real tool usage. In cases where a paper cited multiple publications describing the same tool (e.g., both versions 1.0 and 2.0), we retained only one reference to avoid duplication in citation counts. Non-research articles were included as publications that cite tools without actual use and were accounted for in the final statistics.
We compiled a list of freely available pathway analysis tools capable of performing multi-omics analysis (Table 1). The final list included 28 tools described in 36 papers. Using PubMed, we retrieved all open-access publications citing each tool, resulting in 18,447 full-text articles. After excluding papers for which the publisher did not permit full-text download and papers citing multiple publications describing the same tool, the final dataset comprised 16,285 articles. The earliest citing paper dates to 2006, and the most recent to 2024.
| Tool name | References | Avg citations | Tool usage fraction, % |
|---|---|---|---|
| Gene-level, p-value-based approaches | |||
| KaPPA-View | [14] | 1.2 | 0 |
| MAPE | [15] | 1.8 | 5 |
| INMEX | [16] | 7 | 64.5 |
| 3Omics | [17] | 6.6 | 8.9 |
| IncroMAP | [18] | 2.2 | 60 |
| ActivePathways | [5] | 14.6 | 44 |
| mitch | [19] | 5 | 75 |
| iODA | [20] | 1.25 | 0 |
| Pathway-level, p-value-based approaches | |||
| IMPaLA | [21] | 15.2 | 54.5 |
| iPEAP | [22] | 0.6 | 0 |
| MarVis-Pathway | [23] | 2.2 | 60 |
| BLMA | [24] | 1 | 30 |
| GeneTrail | [12] | 17.8 | 69.3 |
| Mergeomics | [10,11] | 11.6 | 39.3 |
| Pathview | [3,25] | 152.4 | 82.4 |
| PaintOmics | [4,26,27] | 19.6 | 19.3 |
| ReactomeGSA | [28,29] | 22.4 | 50.2 |
| multiGSEA | [30] | 10.8 | 43.1 |
| pathwayMultiomics | [31] | 2.7 | 0 |
| CPA | [32] | 3 | 25 |
| clusterProfiler | [1,33] | 2684.8 | 90.1 |
| Graph transformation-based approaches | |||
| PARADIGM | [2] | 19.4 | 18.4 |
| Subpathway-GM | [34] | 4.2 | 20.2 |
| microGraphite | [35] | 1.4 | 20 |
| MOSClip | [36] | 2 | 8.3 |
| IMPRes-Pro | [37] | 0.8 | 30 |
| Machine-learning-based approaches | |||
| GSOA | [38] | 0.6 | 0 |
| pathwayPCA | [39] | 1.4 | 20 |
Two bibliometric indicators were employed: number of citations per year, representing the yearly count of papers citing the tool’s primary publication(s), and (2) fraction of papers actually employing the tool per year, defined as the ratio of research articles citing the tool in methodological sections to the total number of citing papers for the same year. The first metric captures the temporal dynamics of a tool’s visibility in the literature, while the second reflects its real-world adoption in experimental workflows.
Citation collection and analysis were performed using custom Python and R scripts, which are freely available upon request from the authors.
RESULTS
We have analyzed 16,285 full-text articles citing 28 multi-omics pathway tools, see Materials and Methods and Figure 1.
Citation Per Year
ClusterProfiler tool led with 14,346 citations, while IMPRes-PRO had the fewest at 4 citations. Following the classification proposed by Maghsoudi et al.,[6] we categorized pathway analysis tools into four groups and visualized corresponding citation trends, see Figure 2.
The first group is represented with gene-level, p-value-based integrative approaches and includes a total of 8 tools. Most tools in this group like INMEX, 3Omics, IncroMAP and mitch have modest citation rates of 2-7 per year (Figure 2a). The exception is the ActivePathways tool, originally developed in 2020, that has a higher citation level with 14.6 citations per year for the last five years. Other tools in this group, including KaPPA-View, MAPE, and iODA, are cited on average less than 2 times per year.
The second group of tools (Figure 2b) is represented with pathway-level, p-value-based integrative approaches. This is the largest group, comprising 13 tools. The following tools show similar citation level, not exceeding 25 citations per year for the last five years: IMPaLA, GeneTrail, Mergeomics, Pathview, PaintOmics, ReactomeGSA, MultiGSEA, CPA. Less frequently cited tools, including iPEAP, MarVis-Pathway, BLMA, and pathwayMultiomics, average fewer than 3 citations annually.
Two tools in this group exhibit exceptional citation patterns (Figure 2c). The first one, Pathview, available both as a standalone R-package[3] and its Web-based version[25] averages 152 citations per year over the past five years. The second tool, the ClusterProfiler R-package[1, 33] is the absolute leader among all other tools with 2685 citations per year. However, citation growth for both tools has plateaued over the last two years.
The third group of tools (Figure 2d) consists of graph transformation-based approaches: PARADIGM with 19.4 citations per year and Subpathway-GM with 4.2 citations per year. The low-cited tools include microGraphite, MOSClip, and IMPRes-Pro with fewer than two citations per year.
The fourth group comprises two machine-learning-based tools, both low-cited: PathwayPCA with 1.4 citations per year and GSOA with 0.6 citations per year.
Overall, the two most cited tools for multi-omics pathway analysis (PathView and ClusterProfiler) belong to the pathway-level p-value-based approaches. Furthermore, we do not observe any method for pathway analysis showing a significant increase in the number of citations for the last five years. Both Pathview and ClusterProfiler peaked in citation activity by 2022 and have reached a plateau since then.
Fraction of papers actually employing the tool
We then turned to the assessment of the actual usage of pathway analysis tools in multi-omics research. For each open-access publication citing one of the tools, we determined whether the tool was cited in the Methods section (indicating actual tool employment), or elsewhere, such as in the Introduction or Discussion, where tools are often mentioned in the context of general literature. The proportion of citations representing real-world tool use for every year is plotted on Figure 3.
Among gene-level, p-value-based tools (Figure 3a), 3Omics exhibited the lowest practical usage over the past five years, with only 9% of citations in Methods sections. Other tools in this category, despite lower citation counts, were actually employed in 60-75% of cases. ActivePathways, the most cited tool in this group, was practically applied in 44% of publications, showing a rising trend in usage.
For the pathway-level, p-value-based integrative approaches (Figure 3b and 3c), the most cited tools, ClusterProfiler and PathView were actually used in 90% and 82% of papers. Other tools in this category were utilized in approximately 50% of cases. Paintomics and CPA showed the lowest practical usage, with only 19% and 25% of citations in Methods sections, respectively.
Graph-transformation-based tools generally demonstrated low practical usage (Figure 3d). For instance, PARADIGM, despite relatively high citation counts, was actually used in only 18% of papers. Subpathway-GM showed a notable decline, dropping from 78% practical usage in 2015-2019 to 20% in 2020-2024.
Validation of Tool Usage Detection Method
We understand that the technique we used to determine whether a tool was actually employed in a study is not bias-free. For example, a citation appearing in the Methods section does not always indicate true usage of the tool. In some cases, tools may be mentioned for conceptual comparison or cited as part of a general methodological background. For instance, Kaever et al.[23] cited the MAPE algorithm[15] in the Methods section as a reference to the idea of meta-analysis, without actually applying MAPE in the study.
To assess the accuracy of our estimate of actual tool adoption, we conducted a manual validation. A total of 100 articles were reviewed to confirm whether the tool was actually employed in the paper. Out of these, 89 cases were correctly identified as either true positives or true negatives. There were 4 false positives, where a citation was incorrectly classified as actual usage, and 7 false negatives, where actual usage was missed. Based on this validation, the accuracy of our method for detecting real tool usage was estimated at 89%.
DISCUSSION
Longitudinal analyses of scientific publications offer a powerful way to reveal trends in biomedical research.[40] Building on this concept, our study systematically examined open-access research articles to assess the employment of multi-omics pathway analysis tools.
We aimed to distinguish between tools that are consistently applied in real-world multi-omics studies and those whose recognition remains largely theoretical. Rather than relying solely on citation counts, we analyzed where in the article each tool was cited, specifically focusing on whether the citation occurred in the Methods section, which we interpret as a proxy for actual use. To the best of our knowledge, this is the first systematic effort to quantitatively assess the gap between citation-based popularity and real-world application of pathway analysis tools in multi-omics studies.
We developed a custom pipeline leveraging full-text articles in NISO-JATS format available from PubMed Central. This approach allowed us to extract structured article metadata, body sections, and inline citations. While there are related efforts such as the OpCitance project,[41] which annotates over 137 million in-text citations across 2 million articles with high precision, its data only covers publications up to 2019. In contrast, our dataset includes articles up to the end of 2024, offering a more current view of tool usage trends.
Our findings highlight a discrepancy between citation frequency and actual tool application. While some tools enjoy high overall citation counts due to their conceptual importance or early entry into the field, this does not always translate to widespread practical use. For example, tools such as ReactomeGSA and PARADIGM, despite receiving considerable attention in the literature, showed relatively low actual usage rates - under 50% and 20%, respectively.
Only two tools, namely ClusterProfiler and Pathview, emerged as clear "workhorses" in the field of multiomics pathway analysis. Both tools are cited in hundreds to thousands of papers annually and are actually employed in over 80% of cases. However, the rate of citation growth for both tools has plateaued over the past two years, suggesting that their adoption may have reached a saturation point. This stabilization may indicate that researchers are beginning to explore newer alternatives as the field evolves.
Our analysis has certain limitations. First, the method we used to detect whether a tool is actually employed in the paper is not perfectly accurate, the manual validation showed an overall accuracy of 89%. Nevertheless, our goal was not the absolute precision but rather the identification of general trends across the field. Second, our analysis is restricted to open-access publications and tools that are publicly available. This could introduce bias by underestimating the presence of paywalled papers or the practical use of proprietary software. Third, we limited our evaluation to the pathway tools that had already gained recognition in the community. The tools included were drawn from a comprehensive review[6] published in 2022, ensuring sufficient citation history for trend analysis. As a result, recently published tools were excluded due to insufficient data for evaluation, regardless of their potential. Finally, although the explored tools support multi-omics, they may have been used for single-omics analysis in some papers.
Our approach could be extended in several ways. For instance, one may exclude papers where at least one author overlaps with the original authors of the tool. This stricter criterion would confirm that the tool is used independently of its developers. Furthermore, our methodology can be extended to analyze other categories of omics-processing software beyond pathway analysis, such as network inference or variant calling.
Overall, we propose that future benchmarking efforts include both bibliometric indicators and usage-based metrics. Recognizing the tools that are genuinely integrated into research pipelines will support better practices in tool development and dissemination.
CONCLUSION
This study offers a critical look at the adoption of pathway analysis tools in multi-omics research. By distinguishing between citation and practical use, we identify tools that are truly embedded in the current pipelines for omics data interpretation. Our findings aim to guide researchers toward more informed tool selection.
