From 435987000b5d1e751051006412fc37fe10f82614 Mon Sep 17 00:00:00 2001 From: ivanmilevtues Date: Tue, 17 Jun 2025 01:46:52 +0200 Subject: [PATCH] Added high-level diagrams --- .codeboarding/Application Orchestrator.md | 149 ++++++++++ .codeboarding/Data I-O & Utilities.md | 143 ++++++++++ .../Read Preprocessing & Quality Control.md | 267 ++++++++++++++++++ .codeboarding/Reporting & Visualization.md | 187 ++++++++++++ .../Structural Anomaly Processing.md | 125 ++++++++ .codeboarding/on_boarding.md | 179 ++++++++++++ 6 files changed, 1050 insertions(+) create mode 100644 .codeboarding/Application Orchestrator.md create mode 100644 .codeboarding/Data I-O & Utilities.md create mode 100644 .codeboarding/Read Preprocessing & Quality Control.md create mode 100644 .codeboarding/Reporting & Visualization.md create mode 100644 .codeboarding/Structural Anomaly Processing.md create mode 100644 .codeboarding/on_boarding.md diff --git a/.codeboarding/Application Orchestrator.md b/.codeboarding/Application Orchestrator.md new file mode 100644 index 0000000..84f77ea --- /dev/null +++ b/.codeboarding/Application Orchestrator.md @@ -0,0 +1,149 @@ +```mermaid + +graph LR + + Application_Orchestrator["Application Orchestrator"] + + Command_Line_Interface["Command Line Interface"] + + File_Discovery_and_Job_Management["File Discovery and Job Management"] + + Read_Preprocessing_Quality_Control["Read Preprocessing & Quality Control"] + + Debubbling_Algorithm["Debubbling Algorithm"] + + Application_Orchestrator -- "configures using" --> Command_Line_Interface + + Application_Orchestrator -- "manages file processing and parallel execution via" --> File_Discovery_and_Job_Management + + File_Discovery_and_Job_Management -- "delegates data processing to" --> Read_Preprocessing_Quality_Control + + Application_Orchestrator -- "initiates" --> Debubbling_Algorithm + + Read_Preprocessing_Quality_Control -- "consumes output from" --> Debubbling_Algorithm + +``` + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Component Details + + + +The AfterQC pipeline is orchestrated by the Application Orchestrator, which handles command-line argument parsing, discovers FASTQ files for processing, and manages the parallel execution of preprocessing, quality control, and debubbling tasks. It delegates specific functionalities to dedicated components for efficient and modular data analysis. + + + +### Application Orchestrator + +The Application Orchestrator component, embodied by the `after.py` script, serves as the central control unit for the AfterQC pipeline. It is responsible for parsing command-line arguments, discovering input FASTQ files, initiating parallel processing jobs, and coordinating the overall workflow by delegating tasks to the Read Preprocessing & Quality Control and Structural Anomaly Processing components. + + + + + +**Related Classes/Methods**: + + + +- `after.parseCommand` (13:92) + +- `after.processDir` (100:170) + +- `after.processOptions` (172:174) + +- `after.runDebubble` (176:183) + + + + + +### Command Line Interface + +This component is responsible for defining and parsing command-line arguments, allowing users to configure various parameters for the AfterQC tool, such as input/output paths, trimming settings, and quality control thresholds. + + + + + +**Related Classes/Methods**: + + + +- `after.parseCommand` (13:92) + + + + + +### File Discovery and Job Management + +This component automates the discovery of relevant FASTQ files within a specified directory, prepares unique processing options for each identified file set, and manages the creation and execution of multiprocessing jobs to enable parallel processing of sequencing data. + + + + + +**Related Classes/Methods**: + + + +- `after.processDir` (100:170) + +- `util.parseBool` (28:33) + + + + + +### Read Preprocessing & Quality Control + +This component performs the core preprocessing and quality control operations on sequencing data. It applies trimming, assesses base quality, handles barcode sequences, and conducts overlap analysis, generating quality statistics and filtering reads based on user-defined criteria. + + + + + +**Related Classes/Methods**: + + + +- `preprocesser.seqFilter` (140:818) + +- `after.processOptions` (172:174) + + + + + +### Debubbling Algorithm + +This component implements the debubbling algorithm to identify and remove 'bubble' reads from sequencing data, which are often indicative of sequencing errors or artifacts. It processes input directories to generate and utilize bubble circle information for filtering. + + + + + +**Related Classes/Methods**: + + + +- `debubble.debubbleDir` (35:52) + +- `after.runDebubble` (176:183) + +- `preprocesser.seqFilter:loadBubbleCircles` (156:173) + +- `preprocesser.seqFilter:isInBubble` (175:203) + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/Data I-O & Utilities.md b/.codeboarding/Data I-O & Utilities.md new file mode 100644 index 0000000..95ebb52 --- /dev/null +++ b/.codeboarding/Data I-O & Utilities.md @@ -0,0 +1,143 @@ +```mermaid + +graph LR + + FastqReader["FastqReader"] + + FastqWriter["FastqWriter"] + + SequenceAnalysisUtilities["SequenceAnalysisUtilities"] + + GeneralUtilities["GeneralUtilities"] + + FastqReader -- "provides raw sequence data" --> SequenceAnalysisUtilities + + SequenceAnalysisUtilities -- "processes data from" --> FastqReader + + SequenceAnalysisUtilities -- "provides processed sequence data" --> FastqWriter + + FastqWriter -- "receives data from" --> SequenceAnalysisUtilities + +``` + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Component Details + + + +The Data I/O & Utilities component serves as the foundational layer for handling FASTQ sequencing data within the AfterQC project. It provides robust functionalities for reading and writing FASTQ files, supporting various compression formats. Additionally, it offers a suite of general-purpose and sequence-specific utility functions for data manipulation, quality assessment, and sequence analysis, promoting modularity and reusability across the project's data processing workflows. + + + +### FastqReader + +Manages the reading of FASTQ files, supporting gzipped and bzipped formats. It initializes file handles and provides functionality to iterate through reads. It also includes a utility to check if a file is a FASTQ file. + + + + + +**Related Classes/Methods**: + + + +- `fastq.Reader.__init__` (18:30) + +- `fastq.Reader.nextRead` (36:48) + +- `fastq.Reader.isEOF` (50:51) + +- `fastq.isFastq` (6:11) + + + + + +### FastqWriter + +Handles the writing of data to FASTQ files, including support for gzipped output. It provides methods to write individual lines or complete sequence reads and to manage file stream operations like flushing and closing. + + + + + +**Related Classes/Methods**: + + + +- `fastq.Writer.__init__` (62:75) + +- `fastq.Writer.flush` (77:79) + +- `fastq.Writer.close` (81:84) + +- `fastq.Writer.writeLines` (86:92) + +- `fastq.Writer.writeRead` (94:103) + + + + + +### SequenceAnalysisUtilities + +Provides core utility functions for sequence manipulation and analysis, including calculating edit distances, Hamming distances, determining overlaps using various algorithms, and generating reverse complements. It also includes functions for quality score conversion. + + + + + +**Related Classes/Methods**: + + + +- `util.complement` (35:36) + +- `util.qualNum` (38:39) + +- `util.reverseComplement` (41:50) + +- `util.hammingDistance` (55:61) + +- `util.editDistance` (64:82) + +- `util.distance_threshold` (84:85) + +- `util.overlap` (87:88) + +- `util.overlap_ed` (90:154) + +- `util.overlap_hm` (157:211) + + + + + +### GeneralUtilities + +Contains general-purpose helper functions not specific to sequence data or FASTQ files, such as boolean parsing and string reversal. + + + + + +**Related Classes/Methods**: + + + +- `util.parseBool` (28:33) + +- `util.reverse` (52:53) + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/Read Preprocessing & Quality Control.md b/.codeboarding/Read Preprocessing & Quality Control.md new file mode 100644 index 0000000..c47633b --- /dev/null +++ b/.codeboarding/Read Preprocessing & Quality Control.md @@ -0,0 +1,267 @@ +```mermaid + +graph LR + + SequenceFilter["SequenceFilter"] + + FASTQ_IO["FASTQ_IO"] + + QualityControl_Stats["QualityControl_Stats"] + + QC_Reporting["QC_Reporting"] + + BarcodeProcessor["BarcodeProcessor"] + + Core_Utilities["Core_Utilities"] + + FileSystem_Manager["FileSystem_Manager"] + + JSON_Serialization["JSON_Serialization"] + + SequenceFilter -- "orchestrates" --> FASTQ_IO + + SequenceFilter -- "collects_data_from" --> QualityControl_Stats + + SequenceFilter -- "sends_data_to" --> QC_Reporting + + SequenceFilter -- "applies_logic_from" --> BarcodeProcessor + + SequenceFilter -- "leverages" --> Core_Utilities + + SequenceFilter -- "manages_paths_with" --> FileSystem_Manager + + SequenceFilter -- "exports_stats_via" --> JSON_Serialization + + QualityControl_Stats -- "generates_visual_data_for" --> QC_Reporting + +``` + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Component Details + + + +This graph illustrates the architecture of the 'Read Preprocessing & Quality Control' subsystem. The `SequenceFilter` component acts as the central orchestrator, managing the flow of sequencing data from input to filtered output. It interacts with `FASTQ_IO` for file operations, `QualityControl_Stats` for data analysis, and `QC_Reporting` for generating reports. `BarcodeProcessor` and `Core_Utilities` provide specialized functionalities, while `FileSystem_Manager` and `JSON_Serialization` handle system-level tasks. + + + +### SequenceFilter + +The central component responsible for orchestrating the entire sequencing read processing pipeline. It initializes other components, manages file I/O, applies various filtering criteria (barcode, trimming, bubble, length, polyX, low quality, N count), performs overlap detection and error correction, and coordinates quality control data collection and reporting. + + + + + +**Related Classes/Methods**: + + + +- `preprocesser.seqFilter` (140:818) + +- `preprocesser.seqFilter.loadBubbleCircles` (156:173) + +- `preprocesser.seqFilter.isInBubble` (175:203) + +- `preprocesser.seqFilter.writeReads` (205:231) + +- `preprocesser.seqFilter.addFiguresToReport` (784:818) + + + + + +### FASTQ_IO + +Handles the reading of raw sequencing data from input FASTQ files and the writing of processed, filtered, or categorized reads (good, bad, overlap) to corresponding output FASTQ files. It also provides utility for extracting base names from file paths. + + + + + +**Related Classes/Methods**: + + + +- `fastq.Reader` (16:51) + +- `fastq.Writer` (56:103) + +- `fastq.Reader.nextRead` (36:48) + +- `util.getMainName` (full file reference) + + + + + +### QualityControl_Stats + +Performs detailed statistical analysis on sequencing reads, both before and after filtering. It calculates metrics such as read length, base quality, base content, GC content, and kmer distribution, and prepares this data for visualization and reporting. + + + + + +**Related Classes/Methods**: + + + +- `qualitycontrol.QualityControl` (30:407) + +- `qualitycontrol.QualityControl.statFile` (330:356) + +- `qualitycontrol.QualityControl.autoTrim` (358:386) + +- `qualitycontrol.QualityControl.statRead` (72:121) + +- `qualitycontrol.QualityControl.qc` (323:328) + +- `qualitycontrol.QualityControl.squeeze` (58:70) + +- `qualitycontrol.QualityControl.statPlotly` (312:321) + +- `qualitycontrol.QualityControl.errorPlotly` (287:310) + +- `qualitycontrol.QualityControl.overlapPlotly` (271:285) + +- `qualitycontrol.QualityControl.qualityPlotly` (157:181) + +- `qualitycontrol.QualityControl.contentPlotly` (183:207) + +- `qualitycontrol.QualityControl.gcPlotly` (209:222) + +- `qualitycontrol.QualityControl.discontinuityPlotly` (224:235) + +- `qualitycontrol.QualityControl.strandBiasPlotly` (237:268) + + + + + +### QC_Reporting + +Responsible for compiling and generating the final quality control report. It integrates statistical summaries and various plots provided by the QualityControl_Stats component, setting overall run statistics and version information before outputting the comprehensive report. + + + + + +**Related Classes/Methods**: + + + +- `qcreporter.QCReporter` (21:137) + +- `qcreporter.QCReporter.addFigure` (26:27) + +- `qcreporter.QCReporter.setStat` (29:30) + +- `qcreporter.QCReporter.setVersion` (32:33) + +- `qcreporter.QCReporter.output` (35:37) + + + + + +### BarcodeProcessor + +Manages the identification and manipulation of barcode sequences within reads. This includes detecting barcode lengths, moving barcodes to read names, and trimming them from the sequence, particularly for paired-end reads. + + + + + +**Related Classes/Methods**: + + + +- `barcodeprocesser.BarcodeProcessor` (10:200) + +- `barcodeprocesser.detectBarcode` (18:31) + +- `barcodeprocesser.moveBarcodeToName` (33:44) + +- `barcodeprocesser.moveAndTrimPair` (76:81) + + + + + +### Core_Utilities + +Provides a collection of fundamental helper functions essential for sequence processing, such as trimming reads, detecting poly-X sequences, counting low-quality bases or Ns, performing sequence overlap analysis, complementing bases, converting quality scores, and managing error matrices for correction. + + + + + +**Related Classes/Methods**: + + + +- `util.trim` (full file reference) + +- `util.hasPolyX` (full file reference) + +- `util.lowQualityNum` (full file reference) + +- `util.nNumber` (full file reference) + +- `util.overlap` (87:88) + +- `util.complement` (35:36) + +- `util.qualNum` (38:39) + +- `util.changeString` (235:238) + +- `util.init_error_matrix` (full file reference) + +- `util.merge_error_matrix` (full file reference) + +- `util.getOverlap` (full file reference) + + + + + +### FileSystem_Manager + +Handles interactions with the file system, including extracting directory names, constructing file paths, checking for the existence of files or directories, and creating new directories as needed for organizing output data. + + + + + +**Related Classes/Methods**: _None_ + + + +### JSON_Serialization + +Facilitates the conversion of structured statistical data into a JSON string format, enabling the export of comprehensive quality control metrics for external analysis or storage. + + + + + +**Related Classes/Methods**: + + + +- `util.makeDict` (full file reference) + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/Reporting & Visualization.md b/.codeboarding/Reporting & Visualization.md new file mode 100644 index 0000000..76250c9 --- /dev/null +++ b/.codeboarding/Reporting & Visualization.md @@ -0,0 +1,187 @@ +```mermaid + +graph LR + + QCReporter_Main_Output["QCReporter Main Output"] + + Report_Header_Generator["Report Header Generator"] + + Report_Menu_Generator["Report Menu Generator"] + + Report_Summary_Generator["Report Summary Generator"] + + QCReporter_Data_Access["QCReporter Data Access"] + + Report_Figures_Generator["Report Figures Generator"] + + Data_Formatting_Utilities["Data Formatting Utilities"] + + QCReporter_Main_Output -- "orchestrates" --> Report_Header_Generator + + Report_Header_Generator -- "invokes" --> Report_Menu_Generator + + Report_Header_Generator -- "invokes" --> Report_Summary_Generator + + Report_Header_Generator -- "invokes" --> Report_Figures_Generator + + Report_Summary_Generator -- "retrieves data from" --> QCReporter_Data_Access + + Report_Summary_Generator -- "uses" --> Data_Formatting_Utilities + + Report_Menu_Generator -- "uses" --> Data_Formatting_Utilities + + Report_Figures_Generator -- "uses" --> Data_Formatting_Utilities + +``` + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Component Details + + + +The Reporting & Visualization component, primarily qcreporter.py, is dedicated to generating comprehensive and user-friendly quality control reports. It takes the processed results and metrics from the Read Preprocessing & Quality Control component and visualizes them in an understandable format, often producing HTML reports with plots and summary statistics for easy interpretation of sequencing data quality. + + + +### QCReporter Main Output + +This component is responsible for the top-level orchestration of the QC report generation. It opens the output file and calls various sub-components to write different sections of the HTML report. + + + + + +**Related Classes/Methods**: + + + +- `qcreporter.QCReporter.output` (35:37) + + + + + +### Report Header Generator + +This component generates the HTML header section of the QC report, including CSS styles, and then delegates to other components to generate the menu, summary, and figures sections. It also handles the inclusion of Plotly JavaScript. + + + + + +**Related Classes/Methods**: + + + +- `qcreporter.QCReporter.outputHeader` (39:54) + +- `qcreporter.QCReporter.outputCSS` (56:68) + +- `qcreporter.QCReporter.outputPlotly` (70:78) + + + + + +### Report Menu Generator + +This component creates the navigation menu for the QC report, linking to different sections of the report. It utilizes a utility function to format division IDs for the menu links. + + + + + +**Related Classes/Methods**: + + + +- `qcreporter.QCReporter.outputMenu` (80:87) + + + + + +### Report Summary Generator + +This component generates the summary table section of the QC report, displaying key statistics about the sequencing run. It interacts with the data access component to retrieve information and uses formatting utilities for presentation. + + + + + +**Related Classes/Methods**: + + + +- `qcreporter.QCReporter.outputSummary` (100:116) + +- `qcreporter.QCReporter.outputRow` (89:90) + + + + + +### QCReporter Data Access + +This component is responsible for retrieving and processing sequencing-related data from the internal statistics (`self.stat`) to be presented in the report, such as determining if the sequencing is single-end or pair-end. + + + + + +**Related Classes/Methods**: + + + +- `qcreporter.QCReporter.getSequencing` (92:98) + + + + + +### Report Figures Generator + +This component is responsible for embedding various figures, including both static images and interactive Plotly graphs, into the HTML report. It iterates through a list of figures and formats their titles and containers. + + + + + +**Related Classes/Methods**: + + + +- `qcreporter.QCReporter.outputFigures` (118:137) + + + + + +### Data Formatting Utilities + +This component provides general utility functions used across the report generation process for formatting data. This includes converting strings into valid HTML division IDs and formatting numerical values for display. + + + + + +**Related Classes/Methods**: + + + +- `util.formatDivID` (1:200) + +- `util.formatNumber` (1:200) + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/Structural Anomaly Processing.md b/.codeboarding/Structural Anomaly Processing.md new file mode 100644 index 0000000..eff4de9 --- /dev/null +++ b/.codeboarding/Structural Anomaly Processing.md @@ -0,0 +1,125 @@ +```mermaid + +graph LR + + Bubble_Detector["Bubble Detector"] + + Circle_Detector["Circle Detector"] + + Bubble_Processor["Bubble Processor"] + + Debubble_Orchestrator["Debubble Orchestrator"] + + Bubble_Detector -- "utilizes" --> Circle_Detector + + Bubble_Processor -- "invokes" --> Bubble_Detector + + Debubble_Orchestrator -- "utilizes" --> Bubble_Processor + +``` + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Component Details + + + +The Structural Anomaly Processing component focuses on identifying, detecting, and processing specific structural artifacts and anomalies within sequencing data. This involves detecting 'bubbles' and circular DNA/RNA structures, and implementing algorithms for 'debubbling' to refine or remove these artifacts, thereby improving data quality. The main flow involves an orchestrator initiating the process, which then uses a processor to handle data and delegate detection to specialized detectors for bubbles and circles. + + + +### Bubble Detector + +This component is responsible for identifying and detecting 'bubble' type structural anomalies within the input data. It performs filtering, clustering, and then specifically detects circles within the clustered data. + + + + + +**Related Classes/Methods**: + + + +- `bubbledetector.BubbleDetector` (10:360) + +- `bubbledetector.BubbleDetector:detect` (40:72) + +- `bubbledetector.BubbleDetector:detectCircles` (264:289) + + + + + +### Circle Detector + +This component focuses on the identification of 'circle' type structural anomalies. It contains methods to detect circles directly or on edges, and calculates their area and properties. + + + + + +**Related Classes/Methods**: + + + +- `circledetector.CircleDetector` (10:206) + +- `circledetector.CircleDetector:detect` (22:38) + +- `circledetector.CircleDetector:detectDirectly` (168:206) + +- `circledetector.CircleDetector:detectOnEdge` (107:165) + + + + + +### Bubble Processor + +This component orchestrates the processing of sequencing data to identify and characterize 'bubble' anomalies. It handles file input, parallel processing of files, data aggregation, and then delegates the actual bubble detection to the Bubble Detector for each tile. It also manages output and image generation. + + + + + +**Related Classes/Methods**: + + + +- `bubbleprocesser.BubbleProcesser` (13:396) + +- `bubbleprocesser.BubbleProcesser:run` (33:79) + +- `bubbleprocesser.BubbleProcesser:detectBubbleForTile` (133:142) + + + + + +### Debubble Orchestrator + +This is the top-level component responsible for initiating and coordinating the entire 'debubbling' process. It parses command-line arguments, identifies input files, and then delegates the core processing to the Bubble Processor. It also handles the final output of detected circles. + + + + + +**Related Classes/Methods**: + + + +- `debubble.debubbleDir` (35:52) + +- `debubble.main` (64:87) + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file diff --git a/.codeboarding/on_boarding.md b/.codeboarding/on_boarding.md new file mode 100644 index 0000000..e29d03c --- /dev/null +++ b/.codeboarding/on_boarding.md @@ -0,0 +1,179 @@ +```mermaid + +graph LR + + Application_Orchestrator["Application Orchestrator"] + + Data_I_O_Utilities["Data I-O & Utilities"] + + Read_Preprocessing_Quality_Control["Read Preprocessing & Quality Control"] + + Structural_Anomaly_Processing["Structural Anomaly Processing"] + + Reporting_Visualization["Reporting & Visualization"] + + Application_Orchestrator -- "orchestrates" --> Read_Preprocessing_Quality_Control + + Application_Orchestrator -- "initiates" --> Structural_Anomaly_Processing + + Application_Orchestrator -- "uses" --> Data_I_O_Utilities + + Read_Preprocessing_Quality_Control -- "accesses data" --> Data_I_O_Utilities + + Read_Preprocessing_Quality_Control -- "sends results to" --> Reporting_Visualization + + Structural_Anomaly_Processing -- "accesses data" --> Data_I_O_Utilities + + Reporting_Visualization -- "generates reports from" --> Read_Preprocessing_Quality_Control + + Reporting_Visualization -- "uses for output" --> Data_I_O_Utilities + + click Application_Orchestrator href "https://github.com/OpenGene/AfterQC/blob/main/.codeboarding//Application Orchestrator.md" "Details" + + click Data_I_O_Utilities href "https://github.com/OpenGene/AfterQC/blob/main/.codeboarding//Data I-O & Utilities.md" "Details" + + click Read_Preprocessing_Quality_Control href "https://github.com/OpenGene/AfterQC/blob/main/.codeboarding//Read Preprocessing & Quality Control.md" "Details" + + click Structural_Anomaly_Processing href "https://github.com/OpenGene/AfterQC/blob/main/.codeboarding//Structural Anomaly Processing.md" "Details" + + click Reporting_Visualization href "https://github.com/OpenGene/AfterQC/blob/main/.codeboarding//Reporting & Visualization.md" "Details" + +``` + +[![CodeBoarding](https://img.shields.io/badge/Generated%20by-CodeBoarding-9cf?style=flat-square)](https://github.com/CodeBoarding/GeneratedOnBoardings)[![Demo](https://img.shields.io/badge/Try%20our-Demo-blue?style=flat-square)](https://www.codeboarding.org/demo)[![Contact](https://img.shields.io/badge/Contact%20us%20-%20contact@codeboarding.org-lightgrey?style=flat-square)](mailto:contact@codeboarding.org) + + + +## Component Details + + + +The AfterQC project is designed for comprehensive quality control and preprocessing of sequencing data. Its main flow involves an Application Orchestrator that manages the overall pipeline, delegating tasks to components responsible for reading and writing data, performing initial read preprocessing and quality assessment, detecting and processing structural anomalies like bubbles, and finally generating detailed quality control reports for user interpretation. + + + +### Application Orchestrator + +The Application Orchestrator component, primarily after.py, serves as the central control unit for the AfterQC pipeline. It is responsible for parsing command-line arguments, discovering input FASTQ files, initiating parallel processing jobs, and coordinating the overall workflow by delegating tasks to the Read Preprocessing & Quality Control and Structural Anomaly Processing components. + + + + + +**Related Classes/Methods**: + + + +- `after` (1:200) + +- `after.parseCommand` (13:92) + +- `after.processDir` (100:170) + +- `after.processOptions` (172:174) + +- `after.runDebubble` (176:183) + + + + + +### Data I-O & Utilities + +The Data I/O & Utilities component, encompassing fastq.py and util.py, provides fundamental functionalities for handling FASTQ formatted sequencing data, including reading, writing, and manipulation. Additionally, it offers a collection of general-purpose helper functions and common utilities used across various parts of the AfterQC project, promoting code reusability and maintainability. + + + + + +**Related Classes/Methods**: + + + +- `fastq` (1:200) + +- `util` (1:200) + + + + + +### Read Preprocessing & Quality Control + +This component, comprising preprocesser.py, qualitycontrol.py, and barcodeprocesser.py, is responsible for the initial stages of data preparation and core quality assessment. It performs tasks such as trimming, filtering based on quality, N-base content, and sequence length, processes barcodes for demultiplexing, and conducts comprehensive quality analysis including read quality, GC content, base content, and kmer distribution. + + + + + +**Related Classes/Methods**: + + + +- `preprocesser` (1:200) + +- `preprocesser.seqFilter` (140:818) + +- `qualitycontrol` (1:200) + +- `qualitycontrol.qc` (10:200) + +- `barcodeprocesser` (1:200) + +- `barcodeprocesser.BarcodeProcessor` (10:200) + + + + + +### Structural Anomaly Processing + +The Structural Anomaly Processing component, including bubbledetector.py, bubbleprocesser.py, circledetector.py, and debubble.py, focuses on identifying, detecting, and processing specific structural artifacts and anomalies within sequencing data. This involves detecting 'bubbles' and circular DNA/RNA structures, and implementing algorithms for 'debubbling' to refine or remove these artifacts, thereby improving data quality. + + + + + +**Related Classes/Methods**: + + + +- `bubbledetector` (1:200) + +- `bubbleprocesser` (1:200) + +- `circledetector` (1:200) + +- `debubble` (1:200) + +- `debubble.debubbleDir` (35:52) + + + + + +### Reporting & Visualization + +The Reporting & Visualization component, primarily qcreporter.py, is dedicated to generating comprehensive and user-friendly quality control reports. It takes the processed results and metrics from the Read Preprocessing & Quality Control component and visualizes them in an understandable format, often producing HTML reports with plots and summary statistics for easy interpretation of sequencing data quality. + + + + + +**Related Classes/Methods**: + + + +- `qcreporter` (1:200) + +- `qcreporter.report` (10:200) + + + + + + + + + +### [FAQ](https://github.com/CodeBoarding/GeneratedOnBoardings/tree/main?tab=readme-ov-file#faq) \ No newline at end of file