| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
| Name | Name | Last commit date | ||
|---|---|---|---|---|
Predicting the profiles of one modality (e.g. protein abundance) from another (e.g. mRNA expression).
Repository: openproblems-bio/task_predict_modality
Experimental techniques to measure multiple modalities within the same single cell are increasingly becoming available. The demand for these measurements is driven by the promise to provide a deeper insight into the state of a cell. Yet, the modalities are also intrinsically linked. We know that DNA must be accessible (ATAC data) to produce mRNA (expression data), and mRNA in turn is used as a template to produce protein (protein abundance). These processes are regulated often by the same molecules that they produce: for example, a protein may bind DNA to prevent the production of more mRNA. Understanding these regulatory processes would be transformative for synthetic biology and drug target discovery. Any method that can predict a modality from another must have accounted for these regulatory processes, but the demand for multi-modal data shows that this is not trivial.
| Name | Roles | Github | Orcid | |
|---|---|---|---|---|
| Alejandro Granados | author | agranado | ||
| Alex Tong | author | atong01 | ||
| Bastian Rieck | author | Pseudomanifold | ||
| Benjamin Frey | author | benjaminfreyuu | 0009-0004-7649-8340 | |
| Christopher Lance | author | xlancelottx | 0000-0002-1275-9802 | |
| Daniel Burkhardt | author | dburkhardt | ||
| Kai Waldrant | contributor | KaiWaldrant | 0009-0003-8555-1361 | |
| Kaiwen Deng | contributor | nonztalk | dengkw@umich.edu | |
| Louise Deconinck | author | LouiseDck | ||
| Robrecht Cannoodt | author | rcannood | 0000-0003-3641-729X | |
| Vladimir Shitov | author, maintainer | VladimirShitov | 0000-0002-1960-8812 | |
| Xueer Chen | contributor | xuerchen | xc2579@columbia.edu | |
| Jiwei Liu | contributor | daxiongshu | 0000-0002-8799-9763 | jiweil@nvidia.com |
| Marius Lange | contributor | marius1311 | 0000-0002-4846-1266 |
flowchart TB
file_common_dataset_mod1("<a href='https://github.com/openproblems-bio/task_predict_modality#file-format-raw-dataset-rna'>Raw dataset RNA</a>")
file_common_dataset_mod2("<a href='https://github.com/openproblems-bio/task_predict_modality#file-format-raw-dataset-mod2'>Raw dataset mod2</a>")
comp_process_datasets[/"<a href='https://github.com/openproblems-bio/task_predict_modality#component-type-process-dataset'>Process Dataset</a>"/]
file_train_mod1("<a href='https://github.com/openproblems-bio/task_predict_modality#file-format-train-mod1'>Train mod1</a>")
file_train_mod2("<a href='https://github.com/openproblems-bio/task_predict_modality#file-format-train-mod2'>Train mod2</a>")
file_test_mod1("<a href='https://github.com/openproblems-bio/task_predict_modality#file-format-test-mod1'>Test mod1</a>")
file_test_mod2("<a href='https://github.com/openproblems-bio/task_predict_modality#file-format-solution'>Solution</a>")
comp_method[/"<a href='https://github.com/openproblems-bio/task_predict_modality#component-type-method'>Method</a>"/]
comp_method_train[/"<a href='https://github.com/openproblems-bio/task_predict_modality#component-type-train'>Train</a>"/]
comp_control_method[/"<a href='https://github.com/openproblems-bio/task_predict_modality#component-type-control-method'>Control method</a>"/]
file_pretrained_model("<a href='https://github.com/openproblems-bio/task_predict_modality#file-format-pretrained-model'>Pretrained model</a>")
comp_method_predict[/"<a href='https://github.com/openproblems-bio/task_predict_modality#component-type-predict'>Predict</a>"/]
file_prediction("<a href='https://github.com/openproblems-bio/task_predict_modality#file-format-prediction'>Prediction</a>")
comp_metric[/"<a href='https://github.com/openproblems-bio/task_predict_modality#component-type-metric'>Metric</a>"/]
file_score("<a href='https://github.com/openproblems-bio/task_predict_modality#file-format-score'>Score</a>")
file_common_dataset_mod1---comp_process_datasets
file_common_dataset_mod2---comp_process_datasets
comp_process_datasets-->file_train_mod1
comp_process_datasets-->file_train_mod2
comp_process_datasets-->file_test_mod1
comp_process_datasets-->file_test_mod2
file_train_mod1---comp_method
file_train_mod1---comp_method_train
file_train_mod1---comp_control_method
file_train_mod1-.-comp_method_predict
file_train_mod2---comp_method
file_train_mod2---comp_method_train
file_train_mod2---comp_control_method
file_train_mod2-.-comp_method_predict
file_test_mod1---comp_method
file_test_mod1-.-comp_method_train
file_test_mod1---comp_control_method
file_test_mod1---comp_method_predict
file_test_mod2---comp_control_method
file_test_mod2---comp_metric
comp_method-->file_prediction
comp_method_train-->file_pretrained_model
comp_control_method-->file_prediction
file_pretrained_model---comp_method_predict
comp_method_predict-->file_prediction
file_prediction---comp_metric
comp_metric-->file_score
The RNA modality of the raw dataset.
Example file: resources_test/common/openproblems_neurips2021/bmmc_cite/dataset_mod1.h5ad
Format:
AnnData object obs: 'batch', 'cell_type', 'is_train', 'size_factors' var: 'feature_id', 'feature_name', 'hvg', 'hvg_score' obsm: 'gene_activity' layers: 'counts', 'normalized' uns: 'dataset_id', 'dataset_name', 'dataset_url', 'dataset_reference', 'dataset_summary', 'dataset_description', 'dataset_organism', 'normalization_id', 'gene_activity_var_names'
Data structure:
| Slot | Type | Description |
|---|---|---|
| obs["batch"] | string | Batch information. |
| obs["cell_type"] | string | Cell type annotation. Used to balance the subsample of test cells. |
| obs["is_train"] | string | (Optional) Which split the cell belongs to. Cells labelled ‘train’ become the training set, all other cells (e.g. ‘test’, ‘iid_holdout’) become the test set. Optional: when absent, process_dataset holds out a quarter of the batches instead. |
| obs["size_factors"] | double | (Optional) The size factors of the cells prior to normalization. |
| var["feature_id"] | string | Unique identifier for the feature, usually a ENSEMBL gene id. |
| var["feature_name"] | string | (Optional) A human-readable name for the feature, usually a gene symbol. |
| var["hvg"] | boolean | Whether or not the feature is considered to be a ‘highly variable gene’. |
| var["hvg_score"] | double | A score for the feature indicating how highly variable it is. |
| obsm["gene_activity"] | double | (Optional) ATAC gene activity. |
| layers["counts"] | integer | Raw counts. |
| layers["normalized"] | double | Normalized expression values. |
| uns["dataset_id"] | string | A unique identifier for the dataset. |
| uns["dataset_name"] | string | Nicely formatted name. |
| uns["dataset_url"] | string | (Optional) Link to the original source of the dataset. |
| uns["dataset_reference"] | string | (Optional) Bibtex reference of the paper in which the dataset was published. |
| uns["dataset_summary"] | string | Short description of the dataset. |
| uns["dataset_description"] | string | Long description of the dataset. |
| uns["dataset_organism"] | string | (Optional) The organism of the sample in the dataset. |
| uns["normalization_id"] | string | The unique identifier of the normalization method used. |
| uns["gene_activity_var_names"] | string | (Optional) Names of the gene activity matrix. |
The second modality of the raw dataset. Must be an ADT or an ATAC dataset
Example file: resources_test/common/openproblems_neurips2021/bmmc_cite/dataset_mod2.h5ad
Format:
AnnData object obs: 'batch', 'cell_type', 'is_train', 'size_factors' var: 'feature_id', 'feature_name', 'hvg', 'hvg_score' obsm: 'gene_activity' layers: 'counts', 'normalized' uns: 'dataset_id', 'dataset_name', 'dataset_url', 'dataset_reference', 'dataset_summary', 'dataset_description', 'dataset_organism', 'normalization_id', 'gene_activity_var_names'
Data structure:
| Slot | Type | Description |
|---|---|---|
| obs["batch"] | string | Batch information. |
| obs["cell_type"] | string | Cell type annotation. Used to balance the subsample of test cells. |
| obs["is_train"] | string | (Optional) Which split the cell belongs to. Cells labelled ‘train’ become the training set, all other cells (e.g. ‘test’, ‘iid_holdout’) become the test set. Optional: when absent, process_dataset holds out a quarter of the batches instead. |
| obs["size_factors"] | double | (Optional) The size factors of the cells prior to normalization. |
| var["feature_id"] | string | Unique identifier for the feature, usually a ENSEMBL gene id. |
| var["feature_name"] | string | (Optional) A human-readable name for the feature, usually a gene symbol. |
| var["hvg"] | boolean | Whether or not the feature is considered to be a ‘highly variable gene’. |
| var["hvg_score"] | double | A score for the feature indicating how highly variable it is. |
| obsm["gene_activity"] | double | (Optional) ATAC gene activity. |
| layers["counts"] | integer | Raw counts. |
| layers["normalized"] | double | Normalized expression values. |
| uns["dataset_id"] | string | A unique identifier for the dataset. |
| uns["dataset_name"] | string | Nicely formatted name. |
| uns["dataset_url"] | string | (Optional) Link to the original source of the dataset. |
| uns["dataset_reference"] | string | (Optional) Bibtex reference of the paper in which the dataset was published. |
| uns["dataset_summary"] | string | Short description of the dataset. |
| uns["dataset_description"] | string | Long description of the dataset. |
| uns["dataset_organism"] | string | (Optional) The organism of the sample in the dataset. |
| uns["normalization_id"] | string | The unique identifier of the normalization method used. |
| uns["gene_activity_var_names"] | string | (Optional) Names of the gene activity matrix. |
A predict modality dataset processor.
Arguments:
| Name | Type | Description |
|---|---|---|
| --input_mod1 | file | The RNA modality of the raw dataset. |
| --input_mod2 | file | The second modality of the raw dataset. Must be an ADT or an ATAC dataset. |
| --output_train_mod1 | file | (Output) The mod1 expression values of the train cells. |
| --output_train_mod2 | file | (Output) The mod2 expression values of the train cells. |
| --output_test_mod1 | file | (Output) The mod1 expression values of the test cells. |
| --output_test_mod2 | file | (Output) The ground-truth mod2 expression values of the test cells. |
| --seed | integer | (Optional) The seed for determining the train/test split. Default: 1. |
The mod1 expression values of the train cells.
Example file: resources_test/task_predict_modality/openproblems_neurips2021/bmmc_cite/swap/train_mod1.h5ad
Format:
AnnData object obs: 'batch', 'size_factors' var: 'gene_ids', 'hvg', 'hvg_score' obsm: 'gene_activity' layers: 'counts', 'normalized' uns: 'dataset_id', 'common_dataset_id', 'modality', 'dataset_organism', 'normalization_id', 'gene_activity_var_names'
Data structure:
| Slot | Type | Description |
|---|---|---|
| obs["batch"] | string | Batch information. |
| obs["size_factors"] | double | (Optional) The size factors of the cells prior to normalization. |
| var["gene_ids"] | string | (Optional) The gene identifiers (if available). |
| var["hvg"] | boolean | Whether or not the feature is considered to be a ‘highly variable gene’. |
| var["hvg_score"] | double | A score for the feature indicating how highly variable it is. |
| obsm["gene_activity"] | double | (Optional) ATAC gene activity. |
| layers["counts"] | integer | Raw counts. |
| layers["normalized"] | double | Normalized expression values. |
| uns["dataset_id"] | string | A unique identifier for the dataset. |
| uns["common_dataset_id"] | string | (Optional) A common identifier for the dataset. |
| uns["modality"] | string | The modality of the data, e.g. ‘GEX’, ‘ADT’ or ‘ATAC’. Methods use this to pick modality-specific hyperparameters or models. |
| uns["dataset_organism"] | string | (Optional) The organism of the sample in the dataset. |
| uns["normalization_id"] | string | The unique identifier of the normalization method used. |
| uns["gene_activity_var_names"] | string | (Optional) Names of the gene activity matrix. |
The mod2 expression values of the train cells.
Example file: resources_test/task_predict_modality/openproblems_neurips2021/bmmc_cite/swap/train_mod2.h5ad
Format:
AnnData object obs: 'batch', 'size_factors' var: 'gene_ids', 'hvg', 'hvg_score' obsm: 'gene_activity' layers: 'counts', 'normalized' uns: 'dataset_id', 'common_dataset_id', 'modality', 'dataset_organism', 'normalization_id', 'gene_activity_var_names'
Data structure:
| Slot | Type | Description |
|---|---|---|
| obs["batch"] | string | Batch information. |
| obs["size_factors"] | double | (Optional) The size factors of the cells prior to normalization. |
| var["gene_ids"] | string | (Optional) The gene identifiers (if available). |
| var["hvg"] | boolean | Whether or not the feature is considered to be a ‘highly variable gene’. |
| var["hvg_score"] | double | A score for the feature indicating how highly variable it is. |
| obsm["gene_activity"] | double | (Optional) ATAC gene activity. |
| layers["counts"] | integer | Raw counts. |
| layers["normalized"] | double | Normalized expression values. |
| uns["dataset_id"] | string | A unique identifier for the dataset. |
| uns["common_dataset_id"] | string | (Optional) A common identifier for the dataset. |
| uns["modality"] | string | The modality of the data, e.g. ‘GEX’, ‘ADT’ or ‘ATAC’. Methods use this to pick modality-specific hyperparameters or models. |
| uns["dataset_organism"] | string | (Optional) The organism of the sample in the dataset. |
| uns["normalization_id"] | string | The unique identifier of the normalization method used. |
| uns["gene_activity_var_names"] | string | (Optional) Names of the gene activity matrix. |
The mod1 expression values of the test cells.
Example file: resources_test/task_predict_modality/openproblems_neurips2021/bmmc_cite/swap/test_mod1.h5ad
Format:
AnnData object obs: 'batch', 'size_factors' var: 'gene_ids', 'hvg', 'hvg_score' obsm: 'gene_activity' layers: 'counts', 'normalized' uns: 'dataset_id', 'common_dataset_id', 'modality', 'dataset_name', 'dataset_url', 'dataset_reference', 'dataset_summary', 'dataset_description', 'dataset_organism', 'normalization_id', 'gene_activity_var_names'
Data structure:
| Slot | Type | Description |
|---|---|---|
| obs["batch"] | string | Batch information. |
| obs["size_factors"] | double | (Optional) The size factors of the cells prior to normalization. |
| var["gene_ids"] | string | (Optional) The gene identifiers (if available). |
| var["hvg"] | boolean | Whether or not the feature is considered to be a ‘highly variable gene’. |
| var["hvg_score"] | double | A score for the feature indicating how highly variable it is. |
| obsm["gene_activity"] | double | (Optional) ATAC gene activity. |
| layers["counts"] | integer | Raw counts. |
| layers["normalized"] | double | Normalized expression values. |
| uns["dataset_id"] | string | A unique identifier for the dataset. |
| uns["common_dataset_id"] | string | (Optional) A common identifier for the dataset. |
| uns["modality"] | string | The modality of the data, e.g. ‘GEX’, ‘ADT’ or ‘ATAC’. Methods use this to pick modality-specific hyperparameters or models. |
| uns["dataset_name"] | string | Nicely formatted name. |
| uns["dataset_url"] | string | (Optional) Link to the original source of the dataset. |
| uns["dataset_reference"] | string | (Optional) Bibtex reference of the paper in which the dataset was published. |
| uns["dataset_summary"] | string | Short description of the dataset. |
| uns["dataset_description"] | string | Long description of the dataset. |
| uns["dataset_organism"] | string | (Optional) The organism of the sample in the dataset. |
| uns["normalization_id"] | string | The unique identifier of the normalization method used. |
| uns["gene_activity_var_names"] | string | (Optional) Names of the gene activity matrix. |
The ground-truth mod2 expression values of the test cells.
Example file: resources_test/task_predict_modality/openproblems_neurips2021/bmmc_cite/swap/test_mod2.h5ad
Description:
The ground truth against which predictions are scored. Only the metrics and the control methods receive this file; regular methods never see it.
Format:
AnnData object obs: 'batch', 'size_factors' var: 'gene_ids', 'hvg', 'hvg_score' obsm: 'gene_activity' layers: 'counts', 'normalized' uns: 'dataset_id', 'common_dataset_id', 'modality', 'dataset_name', 'dataset_url', 'dataset_reference', 'dataset_summary', 'dataset_description', 'dataset_organism', 'normalization_id', 'gene_activity_var_names'
Data structure:
| Slot | Type | Description |
|---|---|---|
| obs["batch"] | string | Batch information. |
| obs["size_factors"] | double | (Optional) The size factors of the cells prior to normalization. |
| var["gene_ids"] | string | (Optional) The gene identifiers (if available). |
| var["hvg"] | boolean | Whether or not the feature is considered to be a ‘highly variable gene’. |
| var["hvg_score"] | double | A score for the feature indicating how highly variable it is. |
| obsm["gene_activity"] | double | (Optional) ATAC gene activity. |
| layers["counts"] | integer | Raw counts. |
| layers["normalized"] | double | Normalized expression values. |
| uns["dataset_id"] | string | A unique identifier for the dataset. |
| uns["common_dataset_id"] | string | (Optional) A common identifier for the dataset. |
| uns["modality"] | string | The modality of the data, e.g. ‘GEX’, ‘ADT’ or ‘ATAC’. Methods use this to pick modality-specific hyperparameters or models. |
| uns["dataset_name"] | string | Nicely formatted name. |
| uns["dataset_url"] | string | (Optional) Link to the original source of the dataset. |
| uns["dataset_reference"] | string | (Optional) Bibtex reference of the paper in which the dataset was published. |
| uns["dataset_summary"] | string | Short description of the dataset. |
| uns["dataset_description"] | string | Long description of the dataset. |
| uns["dataset_organism"] | string | (Optional) The organism of the sample in the dataset. |
| uns["normalization_id"] | string | The unique identifier of the normalization method used. |
| uns["gene_activity_var_names"] | string | (Optional) Names of the gene activity matrix. |
A regression method.
Arguments:
| Name | Type | Description |
|---|---|---|
| --input_train_mod1 | file | The mod1 expression values of the train cells. |
| --input_train_mod2 | file | The mod2 expression values of the train cells. |
| --input_test_mod1 | file | The mod1 expression values of the test cells. |
| --output | file | (Output) A prediction of the mod2 expression values of the test cells. |
Train a model to predict the expression of one modality from another.
Arguments:
| Name | Type | Description |
|---|---|---|
| --input_train_mod1 | file | The mod1 expression values of the train cells. |
| --input_train_mod2 | file | The mod2 expression values of the train cells. |
| --input_test_mod1 | file | (Optional) The mod1 expression values of the test cells. |
| --output | file | (Output) A pretrained model for predicting the expression of one modality from another. |
Quality control methods for verifying the pipeline.
Arguments:
| Name | Type | Description |
|---|---|---|
| --input_train_mod1 | file | The mod1 expression values of the train cells. |
| --input_train_mod2 | file | The mod2 expression values of the train cells. |
| --input_test_mod1 | file | The mod1 expression values of the test cells. |
| --input_test_mod2 | file | The ground-truth mod2 expression values of the test cells. |
| --output | file | (Output) A prediction of the mod2 expression values of the test cells. |
A pretrained model for predicting the expression of one modality from another.
Example file: model
Make predictions using a trained model.
Arguments:
| Name | Type | Description |
|---|---|---|
| --input_train_mod1 | file | (Optional) The mod1 expression values of the train cells. |
| --input_train_mod2 | file | (Optional) The mod2 expression values of the train cells. |
| --input_test_mod1 | file | The mod1 expression values of the test cells. |
| --input_model | file | A pretrained model for predicting the expression of one modality from another. |
| --output | file | (Output) A prediction of the mod2 expression values of the test cells. |
A prediction of the mod2 expression values of the test cells
Example file: resources_test/task_predict_modality/openproblems_neurips2021/bmmc_cite/swap/prediction.h5ad
Format:
AnnData object layers: 'normalized' uns: 'dataset_id', 'method_id'
Data structure:
| Slot | Type | Description |
|---|---|---|
| layers["normalized"] | double | Predicted normalized expression values. |
| uns["dataset_id"] | string | A unique identifier for the dataset. |
| uns["method_id"] | string | A unique identifier for the method. |
A predict modality metric.
Arguments:
| Name | Type | Description |
|---|---|---|
| --input_prediction | file | A prediction of the mod2 expression values of the test cells. |
| --input_test_mod2 | file | The ground-truth mod2 expression values of the test cells. |
| --output | file | (Output) Metric score file. |
Metric score file
Example file: resources_test/task_predict_modality/openproblems_neurips2021/bmmc_cite/swap/score.h5ad
Format:
AnnData object uns: 'dataset_id', 'method_id', 'metric_ids', 'metric_values'
Data structure:
| Slot | Type | Description |
|---|---|---|
| uns["dataset_id"] | string | A unique identifier for the dataset. |
| uns["method_id"] | string | A unique identifier for the method. |
| uns["metric_ids"] | string | One or more unique metric identifiers. |
| uns["metric_values"] | double | The metric values obtained for the given prediction. Must be of same length as ‘metric_ids’. |
| Back | FazBrowse Home | New Git URL |