Manage biological registries¶
Registries manage the formalized knowledge & experimental design that anchor dry & wetlab work.
In LaminDB, registries are standard SQL tables, equipped with mechanisms that avoid typos & duplicated data.
In addition, LaminDB makes it easy to import records from public biological ontologies, based on plug-in bionty
.
In this notebook, you’ll see how to manage an in-house ontology anchored in public knowledge.
(If you also manage experimental design through registries, you can access all metadata through one API and store it in one simple SQL database.)
Setup¶
Let us create an instance that has bionty
mounted, which extends to include biological registries:
!lamin init --storage ./test-registries --schema bionty
Show code cell output
💡 connected lamindb: testuser1/test-registries
import lamindb as ln
import bionty as bt
ln.connect("test-registries")
ln.settings.verbosity = "info"
💡 connected lamindb: testuser1/test-registries
Let’s pre-populate the cell type registry with a few records:
bt.Organism.from_public(name="human").save()
bt.CellType.from_public(name="T cell").save()
bt.CellType(name="my T cell subtype").save()
Show code cell output
✅ created 1 Organism record from Bionty matching name: 'human'
✅ created 1 CellType record from Bionty matching name: 'T cell'
💡 also saving parents of CellType(uid='22LvKd01', name='T cell', ontology_id='CL:0000084', synonyms='T-lymphocyte|T lymphocyte|T-cell', description='A Type Of Lymphocyte Whose Defining Characteristic Is The Expression Of A T Cell Receptor Complex.', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:45 UTC')
✅ created 1 CellType record from Bionty matching ontology_id: 'CL:0000542'
❗ now recursing through parents: this only happens once, but is much slower than bulk saving
💡 also saving parents of CellType(uid='X6c7osZ5', name='lymphocyte', ontology_id='CL:0000542', description='A Lymphocyte Is A Leukocyte Commonly Found In The Blood And Lymph That Has The Characteristics Of A Large Nucleus, A Neutral Staining Cytoplasm, And Prominent Heterochromatin.', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:45 UTC')
✅ created 1 CellType record from Bionty matching ontology_id: 'CL:0000842'
💡 also saving parents of CellType(uid='2Jgr5Xx4', name='mononuclear cell', ontology_id='CL:0000842', synonyms='mononuclear leukocyte', description='A Leukocyte With A Single Non-Segmented Nucleus In The Mature Form.', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:46 UTC')
✅ created 2 CellType records from Bionty matching ontology_id: 'CL:0000738', 'CL:0000226'
💡 also saving parents of CellType(uid='3VEAlFdi', name='leukocyte', ontology_id='CL:0000738', synonyms='white blood cell|leucocyte', description='An Achromatic Cell Of The Myeloid Or Lymphoid Lineages Capable Of Ameboid Movement, Found In Blood Or Other Tissue.', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:46 UTC')
✅ created 4 CellType records from Bionty matching ontology_id: 'CL:0000255', 'CL:0000219', 'CL:0002242', 'CL:0000988'
💡 also saving parents of CellType(uid='4WnpvUTH', name='eukaryotic cell', ontology_id='CL:0000255', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:47 UTC')
✅ created 1 CellType record from Bionty matching ontology_id: 'CL:0000000'
💡 also saving parents of CellType(uid='2K93w3xO', name='motile cell', ontology_id='CL:0000219', description='A Cell That Moves By Its Own Activities.', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:47 UTC')
💡 also saving parents of CellType(uid='u3sr1Gdf', name='nucleate cell', ontology_id='CL:0002242', description='A Cell Containing At Least One Nucleus.', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:47 UTC')
💡 also saving parents of CellType(uid='4Ilrnj9U', name='hematopoietic cell', ontology_id='CL:0000988', synonyms='haematopoietic cell|hemopoietic cell|haemopoietic cell', description='A Cell Of A Hematopoietic Lineage.', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:47 UTC')
💡 also saving parents of CellType(uid='2cXC7cgF', name='single nucleate cell', ontology_id='CL:0000226', description='A Cell With A Single Nucleus.', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:46 UTC')
❗ records with similar names exist! did you mean to load one of them?
uid | name | ontology_id | abbr | synonyms | description | public_source_id | run_id | created_by_id | updated_at | |
---|---|---|---|---|---|---|---|---|---|---|
id | ||||||||||
1 | 22LvKd01 | T cell | CL:0000084 | None | T-lymphocyte|T lymphocyte|T-cell | A Type Of Lymphocyte Whose Defining Characteri... | 29 | None | 1 | 2024-05-25 15:24:45.203755+00:00 |
2 | X6c7osZ5 | lymphocyte | CL:0000542 | None | None | A Lymphocyte Is A Leukocyte Commonly Found In ... | 29 | None | 1 | 2024-05-25 15:24:45.768171+00:00 |
3 | 2Jgr5Xx4 | mononuclear cell | CL:0000842 | None | mononuclear leukocyte | A Leukocyte With A Single Non-Segmented Nucleu... | 29 | None | 1 | 2024-05-25 15:24:46.249787+00:00 |
4 | 3VEAlFdi | leukocyte | CL:0000738 | None | white blood cell|leucocyte | An Achromatic Cell Of The Myeloid Or Lymphoid ... | 29 | None | 1 | 2024-05-25 15:24:46.811211+00:00 |
5 | 2cXC7cgF | single nucleate cell | CL:0000226 | None | None | A Cell With A Single Nucleus. | 29 | None | 1 | 2024-05-25 15:24:46.811366+00:00 |
Access records in public ontologies¶
We start with a public ontology for cell types.
.public()
gives you access to the ReferenceTable
, a class for accessing public ontologies.
ReferenceTable
provides simple access to standardized public ontologies that Lamin hosts for reliable and performant access. For most ReferenceTable
objects, you can access the underlying ontology through Pronto.
Let’s create a ReferenceTable
object:
public = bt.CellType.public()
public
PublicOntology
Entity: CellType
Organism: all
Source: cl, 2024-02-13
#terms: 2918
We can use it to search the public ontology against cell types:
public.search("gamma delta T cell").head(3)
Show code cell output
ontology_id | definition | synonyms | parents | __ratio__ | |
---|---|---|---|---|---|
name | |||||
gamma-delta T cell | CL:0000798 | A T Cell That Expresses A Gamma-Delta T Cell R... | gammadelta T cell|gamma-delta T-cell|gamma-del... | [CL:0000084] | 100.000000 |
CD27-negative gamma-delta T cell | CL:0002125 | A Circulating Gamma-Delta T Cell That Expresse... | gammadelta-17 cells | [CL:0000800] | 86.486486 |
mature gamma-delta T cell | CL:0000800 | A Gamma-Delta T Cell That Has A Mature Phenoty... | mature gamma-delta T-lymphocyte|mature gamma-d... | [CL:0002419, CL:0000798] | 83.720930 |
And we can also use it to look up cell types with auto-complete:
lookup = public.lookup()
lookup.gamma_delta_t_cell
Show code cell output
CellType(ontology_id='CL:0000798', name='gamma-delta T cell', definition='A T Cell That Expresses A Gamma-Delta T Cell Receptor Complex.', synonyms='gammadelta T cell|gamma-delta T-cell|gamma-delta T lymphocyte|gamma-delta T-lymphocyte', parents=array(['CL:0000084'], dtype=object))
Create records in in-house ontologies¶
We can now create a record for our in-house SQL registry by passing the result of the lookup in the public ontology to the CellType
constructor:
gdt_cell = bt.CellType(lookup.gamma_delta_t_cell)
gdt_cell
Show code cell output
❗ records with similar names exist! did you mean to load one of them?
uid | name | ontology_id | abbr | synonyms | description | public_source_id | run_id | created_by_id | updated_at | |
---|---|---|---|---|---|---|---|---|---|---|
id | ||||||||||
1 | 22LvKd01 | T cell | CL:0000084 | None | T-lymphocyte|T lymphocyte|T-cell | A Type Of Lymphocyte Whose Defining Characteri... | 29 | None | 1 | 2024-05-25 15:24:45.203755+00:00 |
3 | 2Jgr5Xx4 | mononuclear cell | CL:0000842 | None | mononuclear leukocyte | A Leukocyte With A Single Non-Segmented Nucleu... | 29 | None | 1 | 2024-05-25 15:24:46.249787+00:00 |
4 | 3VEAlFdi | leukocyte | CL:0000738 | None | white blood cell|leucocyte | An Achromatic Cell Of The Myeloid Or Lymphoid ... | 29 | None | 1 | 2024-05-25 15:24:46.811211+00:00 |
5 | 2cXC7cgF | single nucleate cell | CL:0000226 | None | None | A Cell With A Single Nucleus. | 29 | None | 1 | 2024-05-25 15:24:46.811366+00:00 |
6 | 4WnpvUTH | eukaryotic cell | CL:0000255 | None | None | None | 29 | None | 1 | 2024-05-25 15:24:47.374869+00:00 |
CellType(uid='1HuNn2EP', name='gamma-delta T cell', ontology_id='CL:0000798', synonyms='gammadelta T cell|gamma-delta T-cell|gamma-delta T lymphocyte|gamma-delta T-lymphocyte', created_by_id=1, public_source_id=29)
(Alternatively, we could construct the gamma delta T cell via from_public()
, which is synonyms-aware.)
bt.CellType.from_public(ontology_id="CL:0000798")
Show code cell output
✅ created 1 CellType record from Bionty matching ontology_id: 'CL:0000798'
CellType(uid='1HuNn2EP', name='gamma-delta T cell', ontology_id='CL:0000798', synonyms='gammadelta T cell|gamma-delta T-cell|gamma-delta T lymphocyte|gamma-delta T-lymphocyte', description='A T Cell That Expresses A Gamma-Delta T Cell Receptor Complex.', created_by_id=1, public_source_id=29)
When we save this record to the registry, logging informs us that we’re also saving parent ontological terms:
gdt_cell.save()
Show code cell output
💡 also saving parents of CellType(uid='1HuNn2EP', name='gamma-delta T cell', ontology_id='CL:0000798', synonyms='gammadelta T cell|gamma-delta T-cell|gamma-delta T lymphocyte|gamma-delta T-lymphocyte', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:49 UTC')
Will I always see parents being saved?
No, this only happens a single time.
If we accidentally save the same record again, it will be recognized that the record and all parents are already in the registry.
If we save another record that has overlapping parents, only new parents will be saved.
View the ontological hierarchy:
gdt_cell.view_parents()
Or access the parents directly:
gdt_cell.parents.df()
Show code cell output
uid | name | ontology_id | abbr | synonyms | description | public_source_id | run_id | created_by_id | updated_at | |
---|---|---|---|---|---|---|---|---|---|---|
id | ||||||||||
1 | 22LvKd01 | T cell | CL:0000084 | None | T-lymphocyte|T lymphocyte|T-cell | A Type Of Lymphocyte Whose Defining Characteri... | 29 | None | 1 | 2024-05-25 15:24:45.203755+00:00 |
You can construct custom hierarchies of terms by specifying parents:
my_celltype = bt.CellType.filter(name="my T cell subtype").one()
my_celltype.parents.add(gdt_cell)
gdt_cell.view_parents(distance=2, with_children=True)
This cell type and all its parents can now be queried & searched in the registry using bionty.CellType.filter
and bionty.CellType.search
.
Load records for values in data sources¶
When accessing data sources, one often encounters bulk references to entities that might be corrupted or standardized using different standardization schemes.
Let’s consider an example based on an AnnData
object, in the cell_type
annotations of this AnnData
object, we find 4 references to cell types:
adata = ln.core.datasets.anndata_with_obs()
adata.obs.cell_type.value_counts()
Show code cell output
T cell 10
hematopoietic stem cell 10
hepatocyte 10
my new cell type 10
Name: cell_type, dtype: int64
We’d like to load the corresponding records in our in-house ontology to annotate a dataset.
To this end, you’ll typically use from_values
, which will both validate & load records that match the values.
cell_types = bt.CellType.from_values(adata.obs.cell_type)
cell_types
Show code cell output
✅ loaded 1 CellType record matching name: 'T cell'
✅ created 2 CellType records from Bionty matching name: 'hematopoietic stem cell', 'hepatocyte'
❗ did not create CellType record for 1 non-validated name: 'my new cell type'
[CellType(uid='22LvKd01', name='T cell', ontology_id='CL:0000084', synonyms='T-lymphocyte|T lymphocyte|T-cell', description='A Type Of Lymphocyte Whose Defining Characteristic Is The Expression Of A T Cell Receptor Complex.', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:45 UTC'),
CellType(uid='2U8xapxu', name='hematopoietic stem cell', ontology_id='CL:0000037', synonyms='hemopoietic stem cell|blood forming stem cell', description='A Stem Cell From Which All Cells Of The Lymphoid And Myeloid Lineages Develop, Including Blood Cells And Cells Of The Immune System. Hematopoietic Stem Cells Lack Cell Markers Of Effector Cells (Lin-Negative). Lin-Negative Is Defined By Lacking One Or More Of The Following Cell Surface Markers: Cd2, Cd3 Epsilon, Cd4, Cd5 ,Cd8 Alpha Chain, Cd11B, Cd14, Cd19, Cd20, Cd56, Ly6G, Ter119.', created_by_id=1, public_source_id=29),
CellType(uid='7hggmgo1', name='hepatocyte', ontology_id='CL:0000182', description='The Main Structural Component Of The Liver. They Are Specialized Epithelial Cells That Are Organized Into Interconnected Plates Called Lobules. Majority Of Cell Population Of Liver, Polygonal In Shape, Arranged In Plates Or Trabeculae Between Sinusoids; May Have Single Nucleus Or Binucleated.', created_by_id=1, public_source_id=29)]
Logging informed us that 3 cell types were validated. Since we loaded these records at the same time, we could readily use them to annotate a dataset.
What happened under-the-hood?
.from_values()
performs the following look ups:
If registry records match the values, load these records
If values match synonyms of registry records, load these records
(
bionty
-only) If no record in the registry matches, attempt to load records from a public reference(
bionty
-only) Same as 3. but based on synonyms
No records will be returned if input field values aren’t mappable.
Example:
celltype_names = [
"gamma-delta T cell", # existing record with the same name
"T lymphocyte", # existing record with synonym
"hepatocyte", # public record with the same name
"HSC", # public record with synonym
"my new cell type", # Not exist in in-house registry, not exist in public reference
]
bionty.CellType.from_values(celltype_names)
This returns records for all names except from “my new cell type”.
If you’d like to add this new value to the registry, do it like so:
my_celltype = bionty.CellType(name="my new cell type")
my_celltype.save()
Alternatively, we can create entries based on ontology ids:
adata.obs.cell_type_id.unique().tolist()
Show code cell output
['CL:0000084', 'CL:0000037', 'CL:0000182', '']
bt.CellType.from_values(adata.obs.cell_type_id, field=bt.CellType.ontology_id)
Show code cell output
✅ loaded 1 CellType record matching ontology_id: 'CL:0000084'
✅ created 2 CellType records from Bionty matching ontology_id: 'CL:0000037', 'CL:0000182'
[CellType(uid='22LvKd01', name='T cell', ontology_id='CL:0000084', synonyms='T-lymphocyte|T lymphocyte|T-cell', description='A Type Of Lymphocyte Whose Defining Characteristic Is The Expression Of A T Cell Receptor Complex.', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:45 UTC'),
CellType(uid='2U8xapxu', name='hematopoietic stem cell', ontology_id='CL:0000037', synonyms='hemopoietic stem cell|blood forming stem cell', description='A Stem Cell From Which All Cells Of The Lymphoid And Myeloid Lineages Develop, Including Blood Cells And Cells Of The Immune System. Hematopoietic Stem Cells Lack Cell Markers Of Effector Cells (Lin-Negative). Lin-Negative Is Defined By Lacking One Or More Of The Following Cell Surface Markers: Cd2, Cd3 Epsilon, Cd4, Cd5 ,Cd8 Alpha Chain, Cd11B, Cd14, Cd19, Cd20, Cd56, Ly6G, Ter119.', created_by_id=1, public_source_id=29),
CellType(uid='7hggmgo1', name='hepatocyte', ontology_id='CL:0000182', description='The Main Structural Component Of The Liver. They Are Specialized Epithelial Cells That Are Organized Into Interconnected Plates Called Lobules. Majority Of Cell Population Of Liver, Polygonal In Shape, Arranged In Plates Or Trabeculae Between Sinusoids; May Have Single Nucleus Or Binucleated.', created_by_id=1, public_source_id=29)]
If we’re happy with cell_types
records, we save them to the registry:
ln.save(cell_types)
Show code cell output
❗ now recursing through parents: this only happens once, but is much slower than bulk saving
💡 also saving parents of CellType(uid='2U8xapxu', name='hematopoietic stem cell', ontology_id='CL:0000037', synonyms='hemopoietic stem cell|blood forming stem cell', description='A Stem Cell From Which All Cells Of The Lymphoid And Myeloid Lineages Develop, Including Blood Cells And Cells Of The Immune System. Hematopoietic Stem Cells Lack Cell Markers Of Effector Cells (Lin-Negative). Lin-Negative Is Defined By Lacking One Or More Of The Following Cell Surface Markers: Cd2, Cd3 Epsilon, Cd4, Cd5 ,Cd8 Alpha Chain, Cd11B, Cd14, Cd19, Cd20, Cd56, Ly6G, Ter119.', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:51 UTC')
✅ created 3 CellType records from Bionty matching ontology_id: 'CL:0008001', 'CL:0000723', 'CL:0011026'
💡 also saving parents of CellType(uid='5M0BT5FC', name='hematopoietic precursor cell', ontology_id='CL:0008001', description='Any Hematopoietic Cell That Is A Precursor Of Some Other Hematopoietic Cell Type.', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:51 UTC')
💡 also saving parents of CellType(uid='2Dvf9ly5', name='somatic stem cell', ontology_id='CL:0000723', description='A Stem Cell That Can Give Rise To Cell Types Of The Body Other Than Those Of The Germ-Line.', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:51 UTC')
✅ created 1 CellType record from Bionty matching ontology_id: 'CL:0000034'
💡 also saving parents of CellType(uid='jxDD8ajD', name='stem cell', ontology_id='CL:0000034', synonyms='animal stem cell', description='A Relatively Undifferentiated Cell That Retains The Ability To Divide And Proliferate Throughout Life To Provide Progenitor Cells That Can Differentiate Into Specialized Cells.', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:52 UTC')
✅ created 1 CellType record from Bionty matching ontology_id: 'CL:0011115'
💡 also saving parents of CellType(uid='5J0ndawv', name='precursor cell', ontology_id='CL:0011115', description='A Cell That, By Division Or Terminal Differentiation, Can Give Rise To Other Cell Types.', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:52 UTC')
💡 also saving parents of CellType(uid='5fX4hLCd', name='progenitor cell', ontology_id='CL:0011026', description='A Precursor Cell That Has A Tendency To Differentiate Into A Specific Type Of Cell. They Are Descendants Of Stem Cells, Only They Are More Constrained In Their Differentiation Potential Or Capacity For Self-Renewal, And Are Often More Limited In Both Senses.', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:51 UTC')
💡 also saving parents of CellType(uid='7hggmgo1', name='hepatocyte', ontology_id='CL:0000182', description='The Main Structural Component Of The Liver. They Are Specialized Epithelial Cells That Are Organized Into Interconnected Plates Called Lobules. Majority Of Cell Population Of Liver, Polygonal In Shape, Arranged In Plates Or Trabeculae Between Sinusoids; May Have Single Nucleus Or Binucleated.', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:51 UTC')
✅ created 2 CellType records from Bionty matching ontology_id: 'CL:0000066', 'CL:0000417'
💡 also saving parents of CellType(uid='68LNvDH7', name='epithelial cell', ontology_id='CL:0000066', synonyms='epitheliocyte', description='A Cell That Is Usually Found In A Two-Dimensional Sheet With A Free Surface. The Cell Has A Cytoskeleton That Allows For Tight Cell To Cell Contact And For Cell Polarity Where Apical Part Is Directed Towards The Lumen And The Basal Part To The Basal Lamina.', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:53 UTC')
💡 also saving parents of CellType(uid='M3aRHlL9', name='endopolyploid cell', ontology_id='CL:0000417', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:53 UTC')
✅ created 1 CellType record from Bionty matching ontology_id: 'CL:0000412'
💡 also saving parents of CellType(uid='1035XQsH', name='polyploid cell', ontology_id='CL:0000412', description='A Cell That Contains More Than Two Haploid Sets Of Chromosomes.', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:53 UTC')
Now, let’s inspect our in-house registry:
bt.CellType.df()
Show code cell output
uid | name | ontology_id | abbr | synonyms | description | public_source_id | run_id | created_by_id | updated_at | |
---|---|---|---|---|---|---|---|---|---|---|
id | ||||||||||
22 | 1035XQsH | polyploid cell | CL:0000412 | None | None | A Cell That Contains More Than Two Haploid Set... | 29.0 | None | 1 | 2024-05-25 15:24:53.804293+00:00 |
21 | M3aRHlL9 | endopolyploid cell | CL:0000417 | None | None | None | 29.0 | None | 1 | 2024-05-25 15:24:53.205099+00:00 |
20 | 68LNvDH7 | epithelial cell | CL:0000066 | None | epitheliocyte | A Cell That Is Usually Found In A Two-Dimensio... | 29.0 | None | 1 | 2024-05-25 15:24:53.204950+00:00 |
19 | 5J0ndawv | precursor cell | CL:0011115 | None | None | A Cell That, By Division Or Terminal Different... | 29.0 | None | 1 | 2024-05-25 15:24:52.701730+00:00 |
18 | jxDD8ajD | stem cell | CL:0000034 | None | animal stem cell | A Relatively Undifferentiated Cell That Retain... | 29.0 | None | 1 | 2024-05-25 15:24:52.090949+00:00 |
17 | 5fX4hLCd | progenitor cell | CL:0011026 | None | None | A Precursor Cell That Has A Tendency To Differ... | 29.0 | None | 1 | 2024-05-25 15:24:51.609438+00:00 |
16 | 2Dvf9ly5 | somatic stem cell | CL:0000723 | None | None | A Stem Cell That Can Give Rise To Cell Types O... | 29.0 | None | 1 | 2024-05-25 15:24:51.609277+00:00 |
15 | 5M0BT5FC | hematopoietic precursor cell | CL:0008001 | None | None | Any Hematopoietic Cell That Is A Precursor Of ... | 29.0 | None | 1 | 2024-05-25 15:24:51.609133+00:00 |
14 | 7hggmgo1 | hepatocyte | CL:0000182 | None | None | The Main Structural Component Of The Liver. Th... | 29.0 | None | 1 | 2024-05-25 15:24:51.012960+00:00 |
13 | 2U8xapxu | hematopoietic stem cell | CL:0000037 | None | hemopoietic stem cell|blood forming stem cell | A Stem Cell From Which All Cells Of The Lympho... | 29.0 | None | 1 | 2024-05-25 15:24:51.012800+00:00 |
12 | 1HuNn2EP | gamma-delta T cell | CL:0000798 | None | gammadelta T cell|gamma-delta T-cell|gamma-del... | None | 29.0 | None | 1 | 2024-05-25 15:24:49.228238+00:00 |
11 | 2eNGzkOc | my T cell subtype | None | None | None | None | NaN | None | 1 | 2024-05-25 15:24:47.938469+00:00 |
10 | 4bKGljt0 | cell | CL:0000000 | None | None | A Material Entity Of Anatomical Origin (Part O... | 29.0 | None | 1 | 2024-05-25 15:24:47.836592+00:00 |
9 | 4Ilrnj9U | hematopoietic cell | CL:0000988 | None | haematopoietic cell|hemopoietic cell|haemopoie... | A Cell Of A Hematopoietic Lineage. | 29.0 | None | 1 | 2024-05-25 15:24:47.375295+00:00 |
8 | u3sr1Gdf | nucleate cell | CL:0002242 | None | None | A Cell Containing At Least One Nucleus. | 29.0 | None | 1 | 2024-05-25 15:24:47.375159+00:00 |
7 | 2K93w3xO | motile cell | CL:0000219 | None | None | A Cell That Moves By Its Own Activities. | 29.0 | None | 1 | 2024-05-25 15:24:47.375019+00:00 |
6 | 4WnpvUTH | eukaryotic cell | CL:0000255 | None | None | None | 29.0 | None | 1 | 2024-05-25 15:24:47.374869+00:00 |
5 | 2cXC7cgF | single nucleate cell | CL:0000226 | None | None | A Cell With A Single Nucleus. | 29.0 | None | 1 | 2024-05-25 15:24:46.811366+00:00 |
4 | 3VEAlFdi | leukocyte | CL:0000738 | None | white blood cell|leucocyte | An Achromatic Cell Of The Myeloid Or Lymphoid ... | 29.0 | None | 1 | 2024-05-25 15:24:46.811211+00:00 |
3 | 2Jgr5Xx4 | mononuclear cell | CL:0000842 | None | mononuclear leukocyte | A Leukocyte With A Single Non-Segmented Nucleu... | 29.0 | None | 1 | 2024-05-25 15:24:46.249787+00:00 |
2 | X6c7osZ5 | lymphocyte | CL:0000542 | None | None | A Lymphocyte Is A Leukocyte Commonly Found In ... | 29.0 | None | 1 | 2024-05-25 15:24:45.768171+00:00 |
1 | 22LvKd01 | T cell | CL:0000084 | None | T-lymphocyte|T lymphocyte|T-cell | A Type Of Lymphocyte Whose Defining Characteri... | 29.0 | None | 1 | 2024-05-25 15:24:45.203755+00:00 |
Access records in in-house ontologies¶
Search:
bt.CellType.search("gamma delta T cell").df().head(2)
Show code cell output
uid | name | ontology_id | abbr | synonyms | description | public_source_id | run_id | created_by_id | updated_at | |
---|---|---|---|---|---|---|---|---|---|---|
id | ||||||||||
1 | 22LvKd01 | T cell | CL:0000084 | None | T-lymphocyte|T lymphocyte|T-cell | A Type Of Lymphocyte Whose Defining Characteri... | 29.0 | None | 1 | 2024-05-25 15:24:45.203755+00:00 |
3 | 2Jgr5Xx4 | mononuclear cell | CL:0000842 | None | mononuclear leukocyte | A Leukocyte With A Single Non-Segmented Nucleu... | 29.0 | None | 1 | 2024-05-25 15:24:46.249787+00:00 |
Or look up with auto-complete:
cell_types = bt.CellType.lookup()
hsc_record = cell_types.hematopoietic_stem_cell
hsc_record
Show code cell output
CellType(uid='2U8xapxu', name='hematopoietic stem cell', ontology_id='CL:0000037', synonyms='hemopoietic stem cell|blood forming stem cell', description='A Stem Cell From Which All Cells Of The Lymphoid And Myeloid Lineages Develop, Including Blood Cells And Cells Of The Immune System. Hematopoietic Stem Cells Lack Cell Markers Of Effector Cells (Lin-Negative). Lin-Negative Is Defined By Lacking One Or More Of The Following Cell Surface Markers: Cd2, Cd3 Epsilon, Cd4, Cd5 ,Cd8 Alpha Chain, Cd11B, Cd14, Cd19, Cd20, Cd56, Ly6G, Ter119.', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:51 UTC')
Validate & standardize¶
Simple validation of an iterable of values works like so:
bt.CellType.validate(["HSC", "blood forming stem cell"])
Show code cell output
❗ 2 terms (100.00%) are not validated for name: HSC, blood forming stem cell
array([False, False])
Because these values don’t comply with the registry, they’re not validated!
You can easily convert these values to validated standardized names based on synonyms like so:
bt.CellType.standardize(["HSC", "blood forming stem cell"])
Show code cell output
💡 standardized 1/2 terms
['HSC', 'hematopoietic stem cell']
Alternatively, you can use .from_values()
, which will only ever create validated records and automatically standardize under-the-hood:
bt.CellType.from_values(["HSC", "blood forming stem cell"])
Show code cell output
✅ loaded 1 CellType record matching synonyms: 'blood forming stem cell'
❗ did not create CellType record for 1 non-validated name: 'HSC'
[CellType(uid='2U8xapxu', name='hematopoietic stem cell', ontology_id='CL:0000037', synonyms='hemopoietic stem cell|blood forming stem cell', description='A Stem Cell From Which All Cells Of The Lymphoid And Myeloid Lineages Develop, Including Blood Cells And Cells Of The Immune System. Hematopoietic Stem Cells Lack Cell Markers Of Effector Cells (Lin-Negative). Lin-Negative Is Defined By Lacking One Or More Of The Following Cell Surface Markers: Cd2, Cd3 Epsilon, Cd4, Cd5 ,Cd8 Alpha Chain, Cd11B, Cd14, Cd19, Cd20, Cd56, Ly6G, Ter119.', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:51 UTC')]
We can also add new synonyms to a record like so:
hsc_record.add_synonym("HSCs")
And when we encounter this synonym as a value, it will now be standardized using synonyms-lookup, and mapped on the correct registry record:
bt.CellType.standardize(["HSCs"])
Show code cell output
💡 standardized 1/1 terms
['hematopoietic stem cell']
A special synonym is .abbr
(short for abbreviation), which has its own field and can be assigned via:
hsc_record.set_abbr("HSC")
You can create a lookup object from the .abbr
field:
cell_types = bt.CellType.lookup("abbr")
hsc = cell_types.hsc
hsc
Show code cell output
CellType(uid='2U8xapxu', name='hematopoietic stem cell', ontology_id='CL:0000037', abbr='HSC', synonyms='HSCs|blood forming stem cell|hemopoietic stem cell|HSC', description='A Stem Cell From Which All Cells Of The Lymphoid And Myeloid Lineages Develop, Including Blood Cells And Cells Of The Immune System. Hematopoietic Stem Cells Lack Cell Markers Of Effector Cells (Lin-Negative). Lin-Negative Is Defined By Lacking One Or More Of The Following Cell Surface Markers: Cd2, Cd3 Epsilon, Cd4, Cd5 ,Cd8 Alpha Chain, Cd11B, Cd14, Cd19, Cd20, Cd56, Ly6G, Ter119.', created_by_id=1, public_source_id=29, updated_at='2024-05-25 15:24:54 UTC')
The same workflow works for all of bionty
’s registries.
Manage registries across organism¶
Most registries are organism-aware, for instance, Gene
:
bt.Gene.from_public(symbol="TCF7", organism="human")
✅ created 1 Gene record from Bionty matching symbol: 'TCF7'
Gene(uid='7IkHKPl0ScQR', symbol='TCF7', ensembl_gene_id='ENSG00000081059', ncbi_gene_ids='6932', biotype='protein_coding', description='transcription factor 7 ', synonyms='TCF-1', created_by_id=1, organism_id=1, public_source_id=11)
Similarly, API calls that interact with multi-organism registries accept a organism
argument, e.g.:
bt.Gene.validate(["TCF7", "ABC1"], organism="human")
Show code cell output
❗ 2 terms (100.00%) are not validated for symbol: TCF7, ABC1
array([False, False])
And when working with the same organism throughout your analysis/workflow, you can omit the organism
argument by configuring it globally:
bt.settings.organism = "mouse"
bt.Gene.from_public(symbol="Ap5b1")
✅ created 1 Gene record from Bionty matching symbol: 'Ap5b1'
Gene(uid='3b8mHb0MRal4', symbol='Ap5b1', ensembl_gene_id='ENSMUSG00000049562', ncbi_gene_ids='381201', biotype='protein_coding', description='adaptor-related protein complex 5, beta 1 subunit ', synonyms='Gm962', created_by_id=1, organism_id=2, public_source_id=15)
Track underlying ontology versions¶
Under-the-hood, source ontology versions are automatically tracked:
bt.PublicSource.filter(currently_used=True).df()
Show code cell output
uid | entity | organism | currently_used | source | source_name | version | url | md5 | source_website | run_id | created_by_id | updated_at | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
id | |||||||||||||
1 | 5Dlc | Organism | vertebrates | True | ensembl | Ensembl | release-112 | https://ftp.ensembl.org/pub/release-112/specie... | 0ec37e77f4bc2d0b0b47c6c62b9f122d | https://www.ensembl.org | None | 1 | 2024-05-25 15:24:41.968790+00:00 |
6 | 2Jzh | Organism | bacteria | True | ensembl | Ensembl | release-57 | https://ftp.ensemblgenomes.ebi.ac.uk/pub/bacte... | ee28510ed5586ea7ab4495717c96efc8 | https://www.ensembl.org | None | 1 | 2024-05-25 15:24:41.969621+00:00 |
7 | 1kdI | Organism | fungi | True | ensembl | Ensembl | release-57 | http://ftp.ensemblgenomes.org/pub/fungi/releas... | dbcde58f4396ab8b2480f7fe9f83df8a | https://www.ensembl.org | None | 1 | 2024-05-25 15:24:41.969774+00:00 |
8 | 2mIM | Organism | metazoa | True | ensembl | Ensembl | release-57 | http://ftp.ensemblgenomes.org/pub/metazoa/rele... | 424636a574fec078a61cbdddb05f9132 | https://www.ensembl.org | None | 1 | 2024-05-25 15:24:41.969927+00:00 |
9 | 2XQ6 | Organism | plants | True | ensembl | Ensembl | release-57 | https://ftp.ensemblgenomes.ebi.ac.uk/pub/plant... | eadaa1f3e527e4c3940c90c7fa5c8bf4 | https://www.ensembl.org | None | 1 | 2024-05-25 15:24:41.970081+00:00 |
10 | 1Vzs | Organism | all | True | ncbitaxon | NCBItaxon Ontology | 2023-06-20 | s3://bionty-assets/df_all__ncbitaxon__2023-06-... | 00d97ba65627f1cd65636d2df22ea76c | https://github.com/obophenotype/ncbitaxon | None | 1 | 2024-05-25 15:24:41.970232+00:00 |
11 | 1hx4 | Gene | human | True | ensembl | Ensembl | release-112 | s3://bionty-assets/df_human__ensembl__release-... | 4ccda4d88720a326737376c534e8446b | https://www.ensembl.org | None | 1 | 2024-05-25 15:24:41.970385+00:00 |
15 | 76FX | Gene | mouse | True | ensembl | Ensembl | release-112 | s3://bionty-assets/df_mouse__ensembl__release-... | 519cf7b8acc3c948274f66f3155a3210 | https://www.ensembl.org | None | 1 | 2024-05-25 15:24:41.970990+00:00 |
19 | 7LW6 | Gene | saccharomyces cerevisiae | True | ensembl | Ensembl | release-112 | s3://bionty-assets/df_saccharomyces cerevisiae... | 11775126b101233525a0a9e2dd64edae | https://www.ensembl.org | None | 1 | 2024-05-25 15:24:41.971594+00:00 |
22 | 7llW | Protein | human | True | uniprot | Uniprot | 2023-03 | s3://bionty-assets/df_human__uniprot__2023-03_... | 1c46e85c6faf5eff3de5b4e1e4edc4d3 | https://www.uniprot.org | None | 1 | 2024-05-25 15:24:41.972045+00:00 |
24 | 5U7J | Protein | mouse | True | uniprot | Uniprot | 2023-03 | s3://bionty-assets/df_mouse__uniprot__2023-03_... | 9d5e9a8225011d3218e10f9bbb96a46c | https://www.uniprot.org | None | 1 | 2024-05-25 15:24:41.972347+00:00 |
26 | 5nkB | CellMarker | human | True | cellmarker | CellMarker | 2.0 | s3://bionty-assets/human_cellmarker_2.0_CellMa... | d565d4a542a5c7e7a06255975358e4f4 | http://bio-bigdata.hrbmu.edu.cn/CellMarker | None | 1 | 2024-05-25 15:24:41.972649+00:00 |
27 | 6AFz | CellMarker | mouse | True | cellmarker | CellMarker | 2.0 | s3://bionty-assets/mouse_cellmarker_2.0_CellMa... | 189586732c63be949e40dfa6a3636105 | http://bio-bigdata.hrbmu.edu.cn/CellMarker | None | 1 | 2024-05-25 15:24:41.972799+00:00 |
28 | 6cbC | CellLine | all | True | clo | Cell Line Ontology | 2022-03-21 | https://data.bioontology.org/ontologies/CLO/su... | ea58a1010b7e745702a8397a526b3a33 | https://bioportal.bioontology.org/ontologies/CLO | None | 1 | 2024-05-25 15:24:41.972949+00:00 |
29 | 3DeN | CellType | all | True | cl | Cell Ontology | 2024-02-13 | http://purl.obolibrary.org/obo/cl/releases/202... | https://obophenotype.github.io/cell-ontology | None | 1 | 2024-05-25 15:24:41.973099+00:00 | |
34 | 1AyH | Tissue | all | True | uberon | Uberon multi-species anatomy ontology | 2024-02-20 | http://purl.obolibrary.org/obo/uberon/releases... | 2048667b5fdf93192384bdf53cafba18 | http://obophenotype.github.io/uberon | None | 1 | 2024-05-25 15:24:41.973874+00:00 |
39 | LoCG | Disease | all | True | mondo | Mondo Disease Ontology | 2024-02-06 | http://purl.obolibrary.org/obo/mondo/releases/... | 78914fa236773c5ea6605f7570df6245 | https://mondo.monarchinitiative.org | None | 1 | 2024-05-25 15:24:41.974629+00:00 |
44 | 2mou | Disease | human | True | doid | Human Disease Ontology | 2024-01-31 | http://purl.obolibrary.org/obo/doid/releases/2... | b36c15a4610757094f8db64b78ae2693 | https://disease-ontology.org | None | 1 | 2024-05-25 15:24:41.975381+00:00 |
51 | 4usY | ExperimentalFactor | all | True | efo | The Experimental Factor Ontology | 3.63.0 | http://www.ebi.ac.uk/efo/releases/v3.63.0/efo.owl | 603e6f6981d53d501c5921aa3940b095 | https://bioportal.bioontology.org/ontologies/EFO | None | 1 | 2024-05-25 15:24:41.976431+00:00 |
54 | 2WLc | Phenotype | human | True | hp | Human Phenotype Ontology | 2024-03-06 | https://github.com/obophenotype/human-phenotyp... | 36b0d00c24a68edb9131707bc146a4c7 | https://hpo.jax.org | None | 1 | 2024-05-25 15:24:41.976884+00:00 |
58 | 6zE1 | Phenotype | mammalian | True | mp | Mammalian Phenotype Ontology | 2024-02-07 | https://github.com/mgijax/mammalian-phenotype-... | 31c27ed2c7d5774f8b20a77e4e1fd278 | https://github.com/mgijax/mammalian-phenotype-... | None | 1 | 2024-05-25 15:24:41.977501+00:00 |
60 | 7EnA | Phenotype | zebrafish | True | zp | Zebrafish Phenotype Ontology | 2024-01-22 | https://github.com/obophenotype/zebrafish-phen... | 01600a5d392419b27fc567362d4cfff8 | https://github.com/obophenotype/zebrafish-phen... | None | 1 | 2024-05-25 15:24:41.977806+00:00 |
63 | 55lY | Phenotype | all | True | pato | Phenotype And Trait Ontology | 2023-05-18 | http://purl.obolibrary.org/obo/pato/releases/2... | bd472f4971492109493d4ad8a779a8dd | https://github.com/pato-ontology/pato | None | 1 | 2024-05-25 15:24:41.978258+00:00 |
64 | 48aa | Pathway | all | True | go | Gene Ontology | 2023-05-10 | https://data.bioontology.org/ontologies/GO/sub... | e9845499eadaef2418f464cd7e9ac92e | http://geneontology.org | None | 1 | 2024-05-25 15:24:41.978407+00:00 |
67 | 3rm9 | BFXPipeline | all | True | lamin | Bioinformatics Pipeline | 1.0.0 | s3://bionty-assets/bfxpipelines.json | a7eff57a256994692fba46e0199ffc94 | https://lamin.ai | None | 1 | 2024-05-25 15:24:41.978859+00:00 |
68 | 5alK | Drug | all | True | dron | Drug Ontology | 2024-03-02 | https://data.bioontology.org/ontologies/DRON/s... | 84138459de4f65034e979f4e46783747 | https://bioportal.bioontology.org/ontologies/DRON | None | 1 | 2024-05-25 15:24:41.979010+00:00 |
70 | 7CRn | DevelopmentalStage | human | True | hsapdv | Human Developmental Stages | 2020-03-10 | http://aber-owl.net/media/ontologies/HSAPDV/11... | 52181d59df84578ed69214a5cb614036 | https://github.com/obophenotype/developmental-... | None | 1 | 2024-05-25 15:24:41.979316+00:00 |
71 | 16tR | DevelopmentalStage | mouse | True | mmusdv | Mouse Developmental Stages | 2020-03-10 | http://aber-owl.net/media/ontologies/MMUSDV/9/... | 5bef72395d853c7f65450e6c2a1fc653 | https://github.com/obophenotype/developmental-... | None | 1 | 2024-05-25 15:24:41.979465+00:00 |
72 | 3Tlc | Ethnicity | human | True | hancestro | Human Ancestry Ontology | 3.0 | https://github.com/EBISPOT/hancestro/raw/3.0/h... | 76dd9efda9c2abd4bc32fc57c0b755dd | https://github.com/EBISPOT/hancestro | None | 1 | 2024-05-25 15:24:41.982456+00:00 |
73 | 5JnV | BioSample | all | True | ncbi | NCBI BioSample attributes | 2023-09 | s3://bionty-assets/df_all__ncbi__2023-09__BioS... | 918db9bd1734b97c596c67d9654a4126 | https://www.ncbi.nlm.nih.gov/biosample/docs/at... | None | 1 | 2024-05-25 15:24:41.982620+00:00 |
Each record is linked to a versioned public source (if it was created from public):
hepatocyte = bt.CellType.filter(name="hepatocyte").one()
hepatocyte.public_source
Show code cell output
PublicSource(uid='3DeN', entity='CellType', organism='all', currently_used=True, source='cl', source_name='Cell Ontology', version='2024-02-13', url='http://purl.obolibrary.org/obo/cl/releases/2024-02-13/cl.owl', md5='', source_website='https://obophenotype.github.io/cell-ontology', created_by_id=1, updated_at='2024-05-25 15:24:41 UTC')
Create records from specific public ontologies¶
By default, records are created from the "currently_used"
public sources which are configured during the instance initialization, e.g.:
bt.Phenotype.public()
Show code cell output
PublicOntology
Entity: Phenotype
Organism: human
Source: hp, 2024-03-06
#terms: 18697
bt.Phenotype.sources(currently_used=True).df()
Show code cell output
uid | entity | organism | currently_used | source | source_name | version | url | md5 | source_website | run_id | created_by_id | updated_at | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
id | |||||||||||||
54 | 2WLc | Phenotype | human | True | hp | Human Phenotype Ontology | 2024-03-06 | https://github.com/obophenotype/human-phenotyp... | 36b0d00c24a68edb9131707bc146a4c7 | https://hpo.jax.org | None | 1 | 2024-05-25 15:24:41.976884+00:00 |
58 | 6zE1 | Phenotype | mammalian | True | mp | Mammalian Phenotype Ontology | 2024-02-07 | https://github.com/mgijax/mammalian-phenotype-... | 31c27ed2c7d5774f8b20a77e4e1fd278 | https://github.com/mgijax/mammalian-phenotype-... | None | 1 | 2024-05-25 15:24:41.977501+00:00 |
60 | 7EnA | Phenotype | zebrafish | True | zp | Zebrafish Phenotype Ontology | 2024-01-22 | https://github.com/obophenotype/zebrafish-phen... | 01600a5d392419b27fc567362d4cfff8 | https://github.com/obophenotype/zebrafish-phen... | None | 1 | 2024-05-25 15:24:41.977806+00:00 |
63 | 55lY | Phenotype | all | True | pato | Phenotype And Trait Ontology | 2023-05-18 | http://purl.obolibrary.org/obo/pato/releases/2... | bd472f4971492109493d4ad8a779a8dd | https://github.com/pato-ontology/pato | None | 1 | 2024-05-25 15:24:41.978258+00:00 |
Sometimes, the default source doesn’t contain the ontology term you are looking for.
You can then specify to create a record from a non-default source. For instance, instead of using untyped labels for iris organisms as Tutorial: Features & labels, we can use the ncbitaxon
ontology:
public_source = bt.PublicSource.filter(entity="Organism", source="ncbitaxon").one()
iris_setosa = bt.Organism.from_public(name="iris setosa", public_source=public_source)
iris_setosa.save()
Analogously, you can pass public_source
to bulk-create records from a non-default source:
records = bt.Organism.from_values(
["iris setosa", "iris versicolor", "iris virginica"], public_source=public_source
)
ln.save(records)
iris_setosa.parents.get(name="iris").view_parents(with_children=True)
Show code cell content
# clean up test instance
!lamin delete --force test-registries
!rm -r test-registries
💡 deleting instance testuser1/test-registries
rm: cannot remove 'test-registries': No such file or directory