| Title: | Tailored Knowledge Catalog |
|---|---|
| Description: | Facilitate the management of data from knowledge resources that are frequently used alone or together in research environments. In 'TKCat', knowledge resources are manipulated as modeled database (MDB) objects. These objects provide access to the data tables along with a general description of the resource and a detail data model documenting the tables, their fields and their relationships. These MDBs are then gathered in catalogs that can be easily explored an shared. Finally, 'TKCat' provides tools to easily subset, filter and combine MDBs and create new catalogs suited for specific needs. |
| Authors: | Patrice Godard [aut, cre, cph] |
| Maintainer: | Patrice Godard <[email protected]> |
| License: | GPL-3 |
| Version: | 1.1.11 |
| Built: | 2024-09-04 04:31:29 UTC |
| Source: | https://github.com/patzaw/tkcat |
Format bytes numbers in human readable values
.format_bytes(bytes).format_bytes(bytes)
bytes |
a vector of integers |
A vector of character with human readable size
An MDB (Modeled DataBase) based on files: fileMDB
Rename tables of a fileMDB object
## S3 method for class 'chMDB' x$i fileMDB( dataFiles, dbInfo, dataModel, readParameters = DEFAULT_READ_PARAMS, collectionMembers = NULL, check = TRUE, n_max = 10, verbose = FALSE ) ## S3 replacement method for class 'fileMDB' names(x) <- value ## S3 method for class 'fileMDB' rename(.data, ...) ## S3 method for class 'fileMDB' x[i] ## S3 method for class 'fileMDB' x[[i]] ## S3 method for class 'fileMDB' x$i## S3 method for class 'chMDB' x$i fileMDB( dataFiles, dbInfo, dataModel, readParameters = DEFAULT_READ_PARAMS, collectionMembers = NULL, check = TRUE, n_max = 10, verbose = FALSE ) ## S3 replacement method for class 'fileMDB' names(x) <- value ## S3 method for class 'fileMDB' rename(.data, ...) ## S3 method for class 'fileMDB' x[i] ## S3 method for class 'fileMDB' x[[i]] ## S3 method for class 'fileMDB' x$i
x |
a fileMDB object |
i |
the index or the name of the tables to take |
dataFiles |
a named vector of path to data files with
|
dbInfo |
a list with DB information: "name" (only mandatory field), "title", "description", "url", "version", "maintainer". |
dataModel |
a ReDaMoR::RelDataModel object |
readParameters |
a list of parameters for reading the data file.
(e.g. |
collectionMembers |
the members of collections as provided to the collection_members<- function (default: NULL ==> no member). |
check |
logical: if TRUE (default) the data are confronted to the data model |
n_max |
maximum number of records to read
for checks purpose (default: 10). See also |
verbose |
if TRUE display the data confrontation report (default: FALSE) |
value |
new table names |
.data |
a fileMDB object |
... |
Use new_name = old_name to rename selected tables |
A fileMDB object
MDB methods: db_info, data_model, data_tables, collection_members, count_records, dims, filter_with_tables, as_fileMDB
Additional general documentation is related to MDB.
hpof <- read_fileMDB( path=system.file("examples/HPO-subset", package="ReDaMoR"), dataModel=system.file("examples/HPO-model.json", package="ReDaMoR"), dbInfo=list( "name"="HPO", "title"="Data extracted from the HPO database", "description"=paste( "This is a very small subset of the HPO!", "Visit the reference URL for more information" ), "url"="http://human-phenotype-ontology.github.io/" ) ) count_records(hpof) ## The following commands take time on fileMDB object ## Not run: select(hpof, HPO_hp:HPO_diseases) toTake <- "HPO_altId" select(hpof, all_of(toTake)) hpoSlice <- slice(hpof, HPO_diseases=1:10) count_records(hpoSlice) if("stringr" %in% installed.packages()[,"Package"]){ epilHP <- filter( hpof, HPO_diseases=stringr::str_detect( label, stringr::regex("epilepsy", ignore_case=TRUE) ) ) count_records(epilHP) label <- "Rolandic epilepsy" cn <- sym("label") reHP <- filter( hpof, HPO_diseases=!!cn==!!label ) } ## End(Not run)hpof <- read_fileMDB( path=system.file("examples/HPO-subset", package="ReDaMoR"), dataModel=system.file("examples/HPO-model.json", package="ReDaMoR"), dbInfo=list( "name"="HPO", "title"="Data extracted from the HPO database", "description"=paste( "This is a very small subset of the HPO!", "Visit the reference URL for more information" ), "url"="http://human-phenotype-ontology.github.io/" ) ) count_records(hpof) ## The following commands take time on fileMDB object ## Not run: select(hpof, HPO_hp:HPO_diseases) toTake <- "HPO_altId" select(hpof, all_of(toTake)) hpoSlice <- slice(hpof, HPO_diseases=1:10) count_records(hpoSlice) if("stringr" %in% installed.packages()[,"Package"]){ epilHP <- filter( hpof, HPO_diseases=stringr::str_detect( label, stringr::regex("epilepsy", ignore_case=TRUE) ) ) count_records(epilHP) label <- "Rolandic epilepsy" cn <- sym("label") reHP <- filter( hpof, HPO_diseases=!!cn==!!label ) } ## End(Not run)
Add a user to an MDB of a chTKCat object
add_chMDB_user(x, mdb, login, admin = FALSE)add_chMDB_user(x, mdb, login, admin = FALSE)
x |
a chTKCat object |
mdb |
name of the modeled database |
login |
login of the user to drop |
admin |
if the user is an admin of the MDB |
No return value, called for side effects
Import a collection in a chTKCat database
add_chTKCat_collection(x, json, overwrite = FALSE)add_chTKCat_collection(x, json, overwrite = FALSE)
x |
a chTKCat object |
json |
a single character indicating the collection to import. Can be:
|
overwrite |
a logical indicating if the existing collection should be replaced. |
No return value, called for side effects
Add a collection member to an MDB
add_collection_member(x, collection, table, ...)add_collection_member(x, collection, table, ...)
x |
an MDB object |
collection |
a collection title in |
table |
the table providing the collection member |
... |
definition of the collection fields as lists
(e.g. |
Add a feature definition to Knowledge Management Requirements (KMR)
add_feature_def(kmr, name, description, properties)add_feature_def(kmr, name, description, properties)
kmr |
a KMR object |
name |
the name of the feature type |
description |
description of the feature type |
properties |
properties of the feature. A list named with property names. For each property, a list with:
|
The modified KMR MDB object
Add a set of helper functions to a compatible object
## S3 method for class 'MDB' add_helpers(x, code, name, language, kmr, ...) ## S3 method for class 'KMR' add_helpers(x, code, name, language, ...) add_helpers(x, code, name, language, ...)## S3 method for class 'MDB' add_helpers(x, code, name, language, kmr, ...) ## S3 method for class 'KMR' add_helpers(x, code, name, language, ...) add_helpers(x, code, name, language, ...)
x |
an object accepting helpers |
code |
file path to the source code of helper functions |
name |
the name of the helper set |
language |
the programming language of the code (default: "R") |
kmr |
an MDB object with KM requirements |
... |
method specific parameters |
Return x with additional helpers
Add KM feature specifications to an MDB object
add_km_feature(x, kmr, table, feature, fields, unit = as.character(NA))add_km_feature(x, kmr, table, feature, fields, unit = as.character(NA))
x |
an MDB object to update with specification tables |
kmr |
an MDB object with KM requirements |
table |
the name of an existing table in x |
feature |
the name of an existing feature in kmr |
fields |
Either a single character providing the name of an existing field in table or a list named with feature property names from kmr. Each element of the list should provide a "field" slot with the name of the corresponding field and a "unit" slot with the name of the unit if relevant. |
unit |
a single character providing the unit if relevant. Unit information provided in fields override this parameter value. |
An MDB object with additional KM table feature specification
Add empty KM specification tables to an MDB object
add_km_spec(x, kmr)add_km_spec(x, kmr)
x |
an MDB object to update with specification tables |
kmr |
an MDB object with KM requirements |
An MDB object with empty KM specification tables
Add KM table specifications to an MDB object
add_km_table(x, kmr, name, type, features = list())add_km_table(x, kmr, name, type, features = list())
x |
an MDB object to update with specification tables |
kmr |
an MDB object with KM requirements |
name |
the name of an existing table in x |
type |
the name of an existing table type in kmr |
features |
a list of feature definitions. Each element of a list
is a list of parameters for the |
An MDB object with additional KM table specification
Add possible values to an integer or a character feature property in KMR
add_property_values(kmr, feature, property, values)add_property_values(kmr, feature, property, values)
kmr |
a KMR object |
feature |
the name of the feature type |
property |
the name of the property |
values |
a vector of character or integer or a named vector with the description of the values |
The modified KMR MDB object
Add a table definition to Knowledge Management Requirements (KMR)
add_table_def( kmr, name, description, collection = as.character(NA), mandatory_features )add_table_def( kmr, name, description, collection = as.character(NA), mandatory_features )
kmr |
a KMR object |
name |
the name of the table type |
description |
description of the table type |
collection |
the name of the collection of which this table type must be a member (default: NA) |
mandatory_features |
a character vector with mandatory features for this table type |
The modified KMR MDB object
Add possible features to table type in KMR
add_table_features(kmr, table, features)add_table_features(kmr, table, features)
kmr |
a KMR object |
table |
the name of the table type |
features |
a character vector with optional features for this table type |
The modified KMR MDB object
Add a unit definition to Knowledge Management Requirements (KMR)
add_unit_def(kmr, measurement, unit, description)add_unit_def(kmr, measurement, unit, description)
kmr |
an KMR object |
measurement |
the name of the measurement (e.g. "volume") |
unit |
the name of the unit (e.g. "ml") |
description |
a description of the unit (e.g. "milliliter) |
The modified KMR object
Archive a chMDB in a chTKCat
archive_chMDB( x, name, defaultTS = as.POSIXct("1970-01-01 00:00.0", tz = "UTC") )archive_chMDB( x, name, defaultTS = as.POSIXct("1970-01-01 00:00.0", tz = "UTC") )
x |
a chTKCat object |
name |
the name of the database to archive |
defaultTS |
a default timestamp value to use when not existing in
the DB (default: |
No return value, called for side effects
Push an MDB object in a ClickHouse database
as_chMDB(x, tkcon, timestamp = Sys.time(), overwrite = FALSE, by = 10^5)as_chMDB(x, tkcon, timestamp = Sys.time(), overwrite = FALSE, by = 10^5)
x |
an MDB object |
tkcon |
a chTKCat object |
timestamp |
a single POSIXct value as a timestamp for the chMDB instance. The default value is the current system time. If this value is smaller or equal to the chMDB current value, an error is thrown. If NA, the current instance is overwritten (if the overwrite parameter is set to TRUE) without changing the existing timestamp. |
overwrite |
a logical indicating if existing data should be overwritten (default: FALSE) |
by |
the size of the batch: number of records to write together (default: 10^5) |
A chMDB object.
Write an MDB object
## S3 method for class 'chMDB' as_fileMDB( x, path, readParameters = list(delim = "\t", na = "<NA>"), htmlModel = TRUE, compress = TRUE, by = 10^5, ... ) ## S3 method for class 'fileMDB' as_fileMDB( x, path, readParameters = list(delim = "\t", na = "<NA>"), htmlModel = TRUE, compress = TRUE, by = 10^5, ... ) as_fileMDB( x, path, readParameters = list(delim = "\t", na = "<NA>"), htmlModel = TRUE, compress = TRUE, by = 10^5, ... ) ## S3 method for class 'memoMDB' as_fileMDB( x, path, readParameters = list(delim = "\t", na = "<NA>"), htmlModel = TRUE, compress = TRUE, by = 10^5, ... ) ## S3 method for class 'metaMDB' as_fileMDB( x, path, readParameters = list(delim = "\t", na = "<NA>"), htmlModel = TRUE, compress = TRUE, by = 10^5, ... )## S3 method for class 'chMDB' as_fileMDB( x, path, readParameters = list(delim = "\t", na = "<NA>"), htmlModel = TRUE, compress = TRUE, by = 10^5, ... ) ## S3 method for class 'fileMDB' as_fileMDB( x, path, readParameters = list(delim = "\t", na = "<NA>"), htmlModel = TRUE, compress = TRUE, by = 10^5, ... ) as_fileMDB( x, path, readParameters = list(delim = "\t", na = "<NA>"), htmlModel = TRUE, compress = TRUE, by = 10^5, ... ) ## S3 method for class 'memoMDB' as_fileMDB( x, path, readParameters = list(delim = "\t", na = "<NA>"), htmlModel = TRUE, compress = TRUE, by = 10^5, ... ) ## S3 method for class 'metaMDB' as_fileMDB( x, path, readParameters = list(delim = "\t", na = "<NA>"), htmlModel = TRUE, compress = TRUE, by = 10^5, ... )
x |
an MDB object |
path |
the path where the MDB should be written |
readParameters |
The following parameters are currently supported:
|
htmlModel |
a logical. If TRUE (default) the model is also plotted in an html file. |
compress |
a logical specifying whether saving data is to use "gzip" compression (default: TRUE) |
by |
the size of the batch: number of records to write together (default: 10^5) |
... |
method specific parameters |
A fileMDB object.
Convert in a KMR object when possible
as_KMR(x)as_KMR(x)
x |
an object |
A KMR object
Convert any MDB object in a memoMDB object
as_memoMDB(x, ...)as_memoMDB(x, ...)
x |
a MDB object |
... |
additional parameters for the |
A memoMDB object
get_confrontation_report, ReDaMoR::format_confrontation_report and ReDaMoR::format_confrontation_report_md for getting and formatting the report confronting the data to the model.
Insert records by batches in a Clickhouse table
ch_insert(con, dbName, tableName, value, by = 10^6)ch_insert(con, dbName, tableName, value, by = 10^6)
con |
the clickhouse connection |
dbName |
the name of the database |
tableName |
the name of the table |
value |
the table to import |
by |
the size of the batch: number of records to import together (default: 10^6) |
No return value, called for side effects
Change chTKCat password
change_chTKCat_password(x, login, password)change_chTKCat_password(x, login, password)
x |
a chTKCat object |
login |
user login |
password |
new user password |
No return value, called for side effects
Check a chTKCat object
check_chTKCat(x, verbose = FALSE)check_chTKCat(x, verbose = FALSE)
x |
a chTKCat object |
verbose |
a logical indicating if information messages should be displayed. |
Invisible result: chTKCat object
An MDB (Modeled DataBase) relying on ClickHouse: chMDB
Rename tables of a chMDB object
chMDB( tkcon, dbTables, dbInfo, dataModel, collectionMembers = NULL, check = TRUE, n_max = 10, verbose = FALSE ) ## S3 replacement method for class 'chMDB' names(x) <- value ## S3 method for class 'chMDB' rename(.data, ...) ## S3 method for class 'chMDB' x[i] ## S3 method for class 'chMDB' x[[i]]chMDB( tkcon, dbTables, dbInfo, dataModel, collectionMembers = NULL, check = TRUE, n_max = 10, verbose = FALSE ) ## S3 replacement method for class 'chMDB' names(x) <- value ## S3 method for class 'chMDB' rename(.data, ...) ## S3 method for class 'chMDB' x[i] ## S3 method for class 'chMDB' x[[i]]
tkcon |
a chTKCat object |
dbTables |
a named vector of tables in tkcon$chcon with
|
dbInfo |
a list with DB information: "name" (only mandatory field), "title", "description", "url", "version", "maintainer". |
dataModel |
a ReDaMoR::RelDataModel object |
collectionMembers |
the members of collections as provided to the collection_members<- function (default: NULL ==> no member). |
check |
logical: if TRUE (default) the data are confronted to the data model |
n_max |
maximum number of records to read
for checks purpose (default: 10). If 0, the data are not checked.
See also |
verbose |
if TRUE display the data confrontation report |
x |
a chMDB object |
value |
new table names |
.data |
a chMDB object |
... |
Use new_name = old_name to rename selected tables |
i |
the index or the name of the tables to take |
A chMDB object
MDB methods: db_info, data_model, data_tables, collection_members, count_records, dims, filter_with_tables, as_fileMDB
Additional general documentation is related to MDB.
Connect to a ClickHouse TKCat instance
chTKCat( host = "localhost", port = 9111L, user = "default", password, settings = list(max_query_size = 1073741824, use_uncompressed_cache = 0, load_balancing = "random", max_memory_usage = 0, allow_introspection_functions = 1, joined_subquery_requires_alias = 0), ports = NULL, drv = ClickHouseHTTP::ClickHouseHTTP(), ... )chTKCat( host = "localhost", port = 9111L, user = "default", password, settings = list(max_query_size = 1073741824, use_uncompressed_cache = 0, load_balancing = "random", max_memory_usage = 0, allow_introspection_functions = 1, joined_subquery_requires_alias = 0), ports = NULL, drv = ClickHouseHTTP::ClickHouseHTTP(), ... )
host |
a character string specifying the host heberging the database (default: localhost) |
port |
an integer specifying the port on which the database is listening (default: 9111) |
user |
user name |
password |
user password |
settings |
list of Clickhouse settings |
ports |
a named list of available ports for accessing ClickHouse
(default: NULL; example: |
drv |
a DBI driver for connecting to ClickHouse
(default: |
... |
additional parameters for connection (see ClickHouseHTTP::dbConnect,ClickHouseHTTPDriver-method for the default driver) |
a chTKCat object
check_chTKCat(), db_disconnect(), db_reconnect()
Collection members
Collection members
## S3 method for class 'TKCat' collection_members(x, ...) ## S3 method for class 'chMDB' collection_members(x, ...) ## S3 replacement method for class 'chMDB' collection_members(x) <- value ## S3 method for class 'chTKCat' collection_members(x, ...) ## S3 method for class 'fileMDB' collection_members(x, ...) ## S3 replacement method for class 'fileMDB' collection_members(x) <- value collection_members(x, ...) collection_members(x) <- value ## S3 method for class 'memoMDB' collection_members(x, ...) ## S3 replacement method for class 'memoMDB' collection_members(x) <- value ## S3 method for class 'metaMDB' collection_members(x, ...)## S3 method for class 'TKCat' collection_members(x, ...) ## S3 method for class 'chMDB' collection_members(x, ...) ## S3 replacement method for class 'chMDB' collection_members(x) <- value ## S3 method for class 'chTKCat' collection_members(x, ...) ## S3 method for class 'fileMDB' collection_members(x, ...) ## S3 replacement method for class 'fileMDB' collection_members(x) <- value collection_members(x, ...) collection_members(x) <- value ## S3 method for class 'memoMDB' collection_members(x, ...) ## S3 replacement method for class 'memoMDB' collection_members(x) <- value ## S3 method for class 'metaMDB' collection_members(x, ...)
x |
an object with embedded collection members |
... |
names of the collections to focus on. By default, all of them are taken. |
value |
the new collection members. A data.frame with the following columns:
|
A dplyr::tibble with the following columns:
collection (character): The name of the collection
cid (character): Collection identifier
resource (character): The name of the resource
mid (integer): The identifier of the member
table (character): The table recording collection information
field (character): The collection field.
static (logical): TRUE if the field value is common to all elements.
value (character): The name of the table column if static is FALSE or the field value if static is TRUE.
type (character): the type of the field. (not necessarily used ==> NA if not)
Compare two MDB objects
compare_MDB(former, new)compare_MDB(former, new)
former |
an MDB object |
new |
an MDB object |
A tibble with 4 columns:
Information: Compared information
Former: value for the former object
New: value for the new object
Identical: a logical indicating if the 2 values are identical
Count the number of records
## S3 method for class 'MDB' count_records(x, ...) count_records(x, ...)## S3 method for class 'MDB' count_records(x, ...) count_records(x, ...)
x |
an object with embedded data tables |
... |
the name of the tables to consider (default: all of them) |
A named vector with the number of records per table.
Create a database in a chTKCat
create_chMDB(x, name, public = FALSE)create_chMDB(x, name, public = FALSE)
x |
a chTKCat object |
name |
the name of the new database |
public |
if the database data are accessible to any user (default:FALSE) |
No return value, called for side effects
Create a chTKCat user
create_chTKCat_user( x, login, password, contact, admin = FALSE, provider = admin )create_chTKCat_user( x, login, password, contact, admin = FALSE, provider = admin )
x |
a chTKCat object |
login |
user login |
password |
user password (NA ==> no password) |
contact |
contact information (can be NA) |
admin |
a logical indicating if the user is an admin of the chTKCat instance (default: TRUE) |
provider |
a logical indicating if the user is data provider (TRUE) or a data consumer (FALSE: default). If admin is set to TRUE provider will be set to TRUE |
No return value, called for side effects
Create an MDB object with Knowledge Management Requirements (KMR)
create_KMR(name, title, description, version, maintainer)create_KMR(name, title, description, version, maintainer)
name |
the name of the requirements |
title |
the title of the requirements |
description |
the description of the requirements |
version |
version of the requirements |
maintainer |
maintainer of the requirements |
An MDB object with KM requirements data model
Create a piece of knowledge (POK) from an MDB and a KMRobject
create_POK(mdb, kmr, tkcat = NULL)create_POK(mdb, kmr, tkcat = NULL)
mdb |
a MDB object with KM specifications |
kmr |
a KMR object with KM requirements |
tkcat |
A TKCat or chTKCat object to make available in helper environment |
A POK object: a list with 3 slots:
$mdb: the provided MDB object
$kmr: the provided KMR object
$helpers: a list functions to leverage data from mdb and kmr
Get the size of data files from a fileMDB object
data_file_size(x, hr = FALSE)data_file_size(x, hr = FALSE)
x |
a fileMDB object |
hr |
a logical indicating if the values should be "human readable". (default: FALSE) |
a numeric vector with size in bytes (hr=FALSE) or a character vector with size and units (hr=TRUE)
Get the data files from a fileMDB object
data_files(x)data_files(x)
x |
a fileMDB object |
a list with "dataFiles" and "readParameters" for reading the files.
Get object data model
## S3 method for class 'chMDB' data_model(x, ...) ## S3 method for class 'fileMDB' data_model(x, ...) data_model(x, ...) ## S3 method for class 'memoMDB' data_model(x, ...) ## S3 method for class 'metaMDB' data_model(x, rtOnly = FALSE, recursive = FALSE, ...)## S3 method for class 'chMDB' data_model(x, ...) ## S3 method for class 'fileMDB' data_model(x, ...) data_model(x, ...) ## S3 method for class 'memoMDB' data_model(x, ...) ## S3 method for class 'metaMDB' data_model(x, rtOnly = FALSE, recursive = FALSE, ...)
x |
an object with an embedded data model |
... |
method specific parameters |
rtOnly |
if TRUE, the function only returns the relational tables and the corresponding foreign tables (default: FALSE) |
recursive |
if TRUE and rtOnly, the function returns also the relational tables from embedded metaMDBs. |
A ReDaMoR::RelDataModel object
Get object data tables
## S3 method for class 'chMDB' data_tables(x, ..., skip = 0, n_max = Inf) ## S3 method for class 'fileMDB' data_tables(x, ..., skip = 0, n_max = Inf) data_tables(x, ..., skip = 0, n_max = Inf) ## S3 method for class 'memoMDB' data_tables(x, ..., skip = 0, n_max = Inf) ## S3 method for class 'metaMDB' data_tables(x, ..., skip = 0, n_max = Inf)## S3 method for class 'chMDB' data_tables(x, ..., skip = 0, n_max = Inf) ## S3 method for class 'fileMDB' data_tables(x, ..., skip = 0, n_max = Inf) data_tables(x, ..., skip = 0, n_max = Inf) ## S3 method for class 'memoMDB' data_tables(x, ..., skip = 0, n_max = Inf) ## S3 method for class 'metaMDB' data_tables(x, ..., skip = 0, n_max = Inf)
x |
an object with embedded data tables |
... |
the name of the tables to get (default: all of them) |
skip |
the number of rows to skip (default: 0) |
n_max |
maximum number of rows to return (default: Inf) |
A list of dplyr::tibble and matrix
Disconnect an object from a database
## S3 method for class 'chMDB' db_disconnect(x) ## S3 method for class 'chTKCat' db_disconnect(x) db_disconnect(x) ## S3 method for class 'metaMDB' db_disconnect(x)## S3 method for class 'chMDB' db_disconnect(x) ## S3 method for class 'chTKCat' db_disconnect(x) db_disconnect(x) ## S3 method for class 'metaMDB' db_disconnect(x)
x |
an object with a database connection |
No return value, called for side effects
DB information
DB information
## S3 method for class 'chMDB' db_info(x, ...) ## S3 replacement method for class 'chMDB' db_info(x) <- value ## S3 method for class 'fileMDB' db_info(x, ...) ## S3 replacement method for class 'fileMDB' db_info(x) <- value db_info(x, ...) db_info(x) <- value ## S3 method for class 'memoMDB' db_info(x, ...) ## S3 replacement method for class 'memoMDB' db_info(x) <- value ## S3 method for class 'metaMDB' db_info(x, ...) ## S3 replacement method for class 'metaMDB' db_info(x) <- value## S3 method for class 'chMDB' db_info(x, ...) ## S3 replacement method for class 'chMDB' db_info(x) <- value ## S3 method for class 'fileMDB' db_info(x, ...) ## S3 replacement method for class 'fileMDB' db_info(x) <- value db_info(x, ...) db_info(x) <- value ## S3 method for class 'memoMDB' db_info(x, ...) ## S3 replacement method for class 'memoMDB' db_info(x) <- value ## S3 method for class 'metaMDB' db_info(x, ...) ## S3 replacement method for class 'metaMDB' db_info(x) <- value
x |
an object with embedded database information |
... |
method specific parameters |
value |
list with the following elements:
|
A list with the following elements:
name: a single character
title: a single character
description: a single character
url: a single character
version: a single character
maintainer: a single character vector
size: a numeric vector providing the size of the DB in bytes
Reconnect an object to a database
## S3 method for class 'POK' db_reconnect(x, user, password, ntries = 3, ...) ## S3 method for class 'chMDB' db_reconnect(x, user, password, ntries = 3, ...) ## S3 method for class 'chTKCat' db_reconnect(x, user, password, ntries = 3, ...) db_reconnect(x, user, password, ntries = 3, ...) ## S3 method for class 'metaMDB' db_reconnect(x, user, password, ntries = 3, ...)## S3 method for class 'POK' db_reconnect(x, user, password, ntries = 3, ...) ## S3 method for class 'chMDB' db_reconnect(x, user, password, ntries = 3, ...) ## S3 method for class 'chTKCat' db_reconnect(x, user, password, ntries = 3, ...) db_reconnect(x, user, password, ntries = 3, ...) ## S3 method for class 'metaMDB' db_reconnect(x, user, password, ntries = 3, ...)
x |
an object with a database connection |
user |
user name. If not provided, it's taken from x |
password |
user password. If not provided, first the function tries to connect without any password.If it fails, the function asks the user to provide a password. |
ntries |
the number of times the user can enter a wrong password (default: 3) |
... |
additional parameters for methods |
A new database connection object.
Get the DB tables from a chMDB or metaMDB object
db_tables(x, host)db_tables(x, host)
x |
|
host |
the name of host (as returned by |
a list with a chTKCat object (tkcon) and a named vector of DB table names (dbTables).
Decode base64 string
decode_bin(text)decode_bin(text)
text |
the base64 character vector to decode |
One decoded value (e.g. a raw vector corresponding to a binary file)
Detailed information about the format of the tables
## S3 method for class 'chMDB' dims(x, ...) ## S3 method for class 'fileMDB' dims( x, ..., by = 1000, estimateThr = 5e+07, estimateSample = 10^6, showWarnings = TRUE ) dims(x, ...) ## S3 method for class 'memoMDB' dims(x, ...) ## S3 method for class 'metaMDB' dims(x, ...)## S3 method for class 'chMDB' dims(x, ...) ## S3 method for class 'fileMDB' dims( x, ..., by = 1000, estimateThr = 5e+07, estimateSample = 10^6, showWarnings = TRUE ) dims(x, ...) ## S3 method for class 'memoMDB' dims(x, ...) ## S3 method for class 'metaMDB' dims(x, ...)
x |
an object with embedded data tables |
... |
the name of the tables to consider (default: all of them) |
by |
the size of the batch: number of lines to count together (default: 1000) |
estimateThr |
file size threshold in bytes from which an estimation of row number should be computed instead of a precise count (default: 50000000 = 50MB) |
estimateSample |
number of values on which the estimation is based (default: 10^6) |
showWarnings |
a warning is raised by default if estimation is done. |
A tibble with one row for each considered table and the following columns:
name: the name of the table
format: "table", "matrix" or "MatrixMarket"
ncol: number of columns
nrow: number of rows
records: number of records (nrow for tables and ncol*nrow for matrices)
bytes: size in bytes
transposed: FALSE by default. TRUE only for matrices stored in a transposed format.
Drop a database from a chTKCat
drop_chMDB(x, name)drop_chMDB(x, name)
x |
a chTKCat object |
name |
the name of the database to remove |
No return value, called for side effects
Drop a user from a chTKCat object
drop_chTKCat_user(x, login)drop_chTKCat_user(x, login)
x |
a chTKCat object |
login |
login of the user to drop |
No return value, called for side effects
Empty a chMDB in a chTKCat
empty_chMDB(x, name, timestamp = NA)empty_chMDB(x, name, timestamp = NA)
x |
a chTKCat object |
name |
the name of the database to empty |
timestamp |
timestamp of the instance to empty. If NA (default) the current instance is emptied. |
No return value, called for side effects
Encode a binary file in a base64 string
encode_bin(what)encode_bin(what)
what |
a file path or a raw vector |
A character vector of length 1 with the base64 encoded file
Explore available MDB in a shiny web interface
## S3 method for class 'TKCat' explore_MDBs( x, subSetSize = 100, download = FALSE, workers = 4, title = NULL, skinColors = "green", logoDiv = TKCAT_LOGO_DIV, rDirs = NULL, tabTitle = "TKCat", tabIcon = "www/TKCat-small.png", ... ) ## S3 method for class 'chTKCat' explore_MDBs( x, subSetSize = 100, host = x$chcon@host, download = FALSE, workers = 4, userManager = NULL, title = NULL, skinColors = c("blue", "yellow"), logoDiv = TKCAT_LOGO_DIV, tabTitle = "chTKCat", tabIcon = "www/TKCat-small.png", rDirs = NULL, ... ) explore_MDBs(x, ...)## S3 method for class 'TKCat' explore_MDBs( x, subSetSize = 100, download = FALSE, workers = 4, title = NULL, skinColors = "green", logoDiv = TKCAT_LOGO_DIV, rDirs = NULL, tabTitle = "TKCat", tabIcon = "www/TKCat-small.png", ... ) ## S3 method for class 'chTKCat' explore_MDBs( x, subSetSize = 100, host = x$chcon@host, download = FALSE, workers = 4, userManager = NULL, title = NULL, skinColors = c("blue", "yellow"), logoDiv = TKCAT_LOGO_DIV, tabTitle = "chTKCat", tabIcon = "www/TKCat-small.png", rDirs = NULL, ... ) explore_MDBs(x, ...)
x |
|
subSetSize |
the maximum number of records to show |
download |
a logical indicating if data can be downloaded (default: FALSE). If TRUE a temporary directory is created and made available for shiny. |
workers |
number of available workers when download is available (default: 4) |
title |
A title for the application. If NULL (default): the chTKCat instance name |
skinColors |
two colors for the application skin: one for default connection ("blue" by default) and one for user connection ("yellow" by default). Working values: "blue", "black", "purple", "green", "red", "yellow". |
logoDiv |
a shiny::div object with a logo to display in side bar. The default is the TKCat hex sticker with a link to TKCat github repository. |
rDirs |
a named character vector with resource path for shiny::addResourcePath |
tabTitle |
a title to display in tab (default: "chTKCat") |
tabIcon |
a path to an image (in available resource paths: "www", "doc" or in rDirs) to use as a tab icon. |
... |
method specific parameters |
host |
the name of the host to show in the application |
userManager |
URL for user management interface
(see |
No return value, called for side effects
Filter a matrix stored in an MDB
## S3 method for class 'chMDB' filter_mdb_matrix(x, tableName, ...) ## S3 method for class 'fileMDB' filter_mdb_matrix(x, tableName, .by = 10^5, ...) filter_mdb_matrix(x, tableName, ...) ## S3 method for class 'memoMDB' filter_mdb_matrix(x, tableName, ...) ## S3 method for class 'metaMDB' filter_mdb_matrix(x, tableName, ...)## S3 method for class 'chMDB' filter_mdb_matrix(x, tableName, ...) ## S3 method for class 'fileMDB' filter_mdb_matrix(x, tableName, .by = 10^5, ...) filter_mdb_matrix(x, tableName, ...) ## S3 method for class 'memoMDB' filter_mdb_matrix(x, tableName, ...) ## S3 method for class 'metaMDB' filter_mdb_matrix(x, tableName, ...)
x |
an MDB object |
tableName |
a character vector of length 1 corresponding to the name of the table to filter (must be a matrix) |
... |
character vectors with the row names and/or columns names to select. The names of the parameters must correspond to the name of the column and of the row fields (the matrix cannot be filtered from values). |
.by |
the size of the batch: number of lines to process together (default: 10000) |
A sub-matrix of tableName in x. Only existing elements are returned. No error is raised if any element is missing. The result must be checked and adapted to user needs.
## Not run: ## Return the matrix of expression values focused on the selected genes filter_mdb_matrix(x=db, "Expression_value", gene=c("SNCA", "MAPT")) ## End(Not run)## Not run: ## Return the matrix of expression values focused on the selected genes filter_mdb_matrix(x=db, "Expression_value", gene=c("SNCA", "MAPT")) ## End(Not run)
Filter an MDB object according to provided tables
## S3 method for class 'chMDB' filter_with_tables(x, tables, checkTables = TRUE, by = 10^5, ...) ## S3 method for class 'fileMDB' filter_with_tables(x, tables, checkTables = TRUE, by = 10^5, ...) filter_with_tables(x, tables, checkTables = TRUE, ...) ## S3 method for class 'memoMDB' filter_with_tables(x, tables, checkTables = TRUE, ...) ## S3 method for class 'metaMDB' filter_with_tables(x, tables, checkTables = TRUE, ...)## S3 method for class 'chMDB' filter_with_tables(x, tables, checkTables = TRUE, by = 10^5, ...) ## S3 method for class 'fileMDB' filter_with_tables(x, tables, checkTables = TRUE, by = 10^5, ...) filter_with_tables(x, tables, checkTables = TRUE, ...) ## S3 method for class 'memoMDB' filter_with_tables(x, tables, checkTables = TRUE, ...) ## S3 method for class 'metaMDB' filter_with_tables(x, tables, checkTables = TRUE, ...)
x |
an MDB object |
tables |
a named list of tibbles to filter with. The names should correspond to the table names in x and the tibbles should fit the data model. |
checkTables |
if TRUE, the tables are confronted to their model in the data model of x. |
by |
the size of the batch: number of lines to process together (default: 10000) |
... |
method specific parameters |
a memoMDB object
Filter a chMDB object and return a memoMDB
## S3 method for class 'chMDB' filter(.data, ..., by = 10^5, .preserve = FALSE)## S3 method for class 'chMDB' filter(.data, ..., by = 10^5, .preserve = FALSE)
.data |
a chMDB object |
... |
each argument should have the name of one of the tables of the chMDB object and contain a simple logical expression involving the names of the corresponding table. |
by |
the size of the batch: number of records to filter together (default: 10^5) |
.preserve |
not used |
a memoMDB object
Filter a fileMDB object and return a memoMDB
## S3 method for class 'fileMDB' filter(.data, ..., .preserve = FALSE)## S3 method for class 'fileMDB' filter(.data, ..., .preserve = FALSE)
.data |
a fileMDB object |
... |
each argument should have the name of one of the tables of the fileMDB object and contain a simple logical expression involving the names of the corresponding table. |
.preserve |
not used |
a memoMDB object
Filter a memoMDB object
## S3 method for class 'memoMDB' filter(.data, ..., .preserve = FALSE)## S3 method for class 'memoMDB' filter(.data, ..., .preserve = FALSE)
.data |
a memoMDB object |
... |
each argument should have the name of one of the tables of the memoMDB object and contain a simple logical expression involving the names of the corresponding table. |
.preserve |
not used |
a filtered memoMDB object
Filter a metaMDB object
## S3 method for class 'metaMDB' filter(.data, ..., .preserve = FALSE)## S3 method for class 'metaMDB' filter(.data, ..., .preserve = FALSE)
.data |
a metaMDB object |
... |
each argument should have the name of one of the tables of the metaMDB object and contain a simple logical expression involving the names of the corresponding table. |
.preserve |
not used |
a filtered memoMDB object
Format a chTKCat object for printing
## S3 method for class 'chTKCat' format(x, ...)## S3 method for class 'chTKCat' format(x, ...)
x |
a chTKCat object |
... |
not used |
A single character
Get the metadata of an MDB from a chTKCat connection
get_chMDB_metadata(x, dbName, timestamp = NA)get_chMDB_metadata(x, dbName, timestamp = NA)
x |
a chTKCat object |
dbName |
the name of the MDB |
timestamp |
the timestamp of the instance to get. Default=NA: get the current version. |
A list with the following elements:
dbInfo: General information regarding the MDB
dataModel: The data model
collectionMembers: Members of different collections
access: type of access to the MDB
Get instance timestamps of an MDB in chTKCat
get_chMDB_timestamps(x, name)get_chMDB_timestamps(x, name)
x |
a chTKCat object |
name |
the name of the database |
A tibble with the instance "timestamp" and a logical indicating if it's the "current" one or not.
Get a collection from a chTKCat
get_chTKCat_collection(x, title)get_chTKCat_collection(x, title)
x |
a chTKCat object |
title |
the title of the collection to get |
The definition of the collection as a JSON string.
Get the default mapper function for a collection
get_collection_mapper(collection)get_collection_mapper(collection)
collection |
the name of the targeted collection
(it should belong to local collections: see |
A function to map collection members.
Get the last generated MDB confrontation report
get_confrontation_report()get_confrontation_report()
A confrontation report generated by ReDaMoR::confront_data()
Get database hosts
## S3 method for class 'DBIConnection' get_hosts(x, ...) ## S3 method for class 'chMDB' get_hosts(x, ...) ## S3 method for class 'chTKCat' get_hosts(x, ...) get_hosts(x, ...) ## S3 method for class 'metaMDB' get_hosts(x, ...)## S3 method for class 'DBIConnection' get_hosts(x, ...) ## S3 method for class 'chMDB' get_hosts(x, ...) ## S3 method for class 'chTKCat' get_hosts(x, ...) get_hosts(x, ...) ## S3 method for class 'metaMDB' get_hosts(x, ...)
x |
an object with database connection(s) |
... |
additional parameters for methods. |
A character vector with hosts information (generaly 1) in the following shape: "host:port"
Get the json definition of a local collection of concepts
get_local_collection(title)get_local_collection(title)
title |
the title of the collection to get |
The definition of the collection as a JSON string.
Get an MDB object from a TKCat related object
## S3 method for class 'TKCat' get_MDB(x, dbName, ...) ## S3 method for class 'chTKCat' get_MDB(x, dbName, timestamp = NA, check = TRUE, n_max = 10, ...) get_MDB(x, dbName, ...)## S3 method for class 'TKCat' get_MDB(x, dbName, ...) ## S3 method for class 'chTKCat' get_MDB(x, dbName, timestamp = NA, check = TRUE, n_max = 10, ...) get_MDB(x, dbName, ...)
x |
|
dbName |
the name of the database |
... |
method specific parameters |
timestamp |
the timestamp of the instance to get. Default=NA: get the current version. |
check |
logical: if TRUE (default) the data are confronted to the data model |
n_max |
maximum number of records to read
for checks purpose (default: 10). See also |
An MDB object
get_confrontation_report, ReDaMoR::format_confrontation_report and ReDaMoR::format_confrontation_report_md for getting and formatting the report confronting the data to the model.
Get a POK from a chTKCat connection or a TKCat object
get_POK(x, mdb, kmr)get_POK(x, mdb, kmr)
x |
|
mdb |
MDB object with KM specifications or its name in tkcat |
kmr |
KMR object with KM requirements or its name in tkcat |
A POK object
Get SQL query
## S3 method for class 'chMDB' get_query(x, query, autoalias = !is_current_chMDB(x), ...) ## S3 method for class 'chTKCat' get_query(x, query, ...) get_query(x, query, ...)## S3 method for class 'chMDB' get_query(x, query, autoalias = !is_current_chMDB(x), ...) ## S3 method for class 'chTKCat' get_query(x, query, ...) get_query(x, query, ...)
x |
an object with a database connection |
query |
the SQL query |
autoalias |
Change this parameter only if you know what you're doing. if TRUE, make relevant alias to query the chMDB using the table names from the data model. If FALSE, the user must know the table instance name in the remote database. By default, autoalias is set to TRUE when using a non-current instance of the database. |
... |
method specific parameters |
A tibble with query results
Get a set of helper functions from an object
## S3 method for class 'MDB' get_R_helpers(x, hnames = NA, kmr, tkcat = NULL, ...) ## S3 method for class 'KMR' get_R_helpers(x, hnames = NA, tkcat = NULL, mdb = NULL, ...) get_R_helpers(x, hnames, ...)## S3 method for class 'MDB' get_R_helpers(x, hnames = NA, kmr, tkcat = NULL, ...) ## S3 method for class 'KMR' get_R_helpers(x, hnames = NA, tkcat = NULL, mdb = NULL, ...) get_R_helpers(x, hnames, ...)
x |
an object with helpers |
hnames |
the names of the helper sets. If NA (default), all available are sourced. |
kmr |
a KMR object |
tkcat |
A TKCat or chTKCat object to make available in helper environment |
... |
method specific parameters |
mdb |
An MDB object to make available in helper environment |
x, kmr and tkcat objects are made available in helpers environment as 'THISMDB', 'THISKMR' and 'THISTKCAT' objects respectively and can be used as such within helpers code.
x, tkcat and mdb objects are made available in helpers environment as 'THISKMR', 'THISTKCAT' and 'THISMDB' objects respectively and can be used as such within helpers code.
Return a list of functions
Check if KM specifications are available in an MDB object
has_km_spec(x, kmr)has_km_spec(x, kmr)
x |
an MDB object with specification tables |
kmr |
an MDB object with KM requirements |
A logical: TRUE if the MDB has KM specifications
Get the first records of each object data tables
## S3 method for class 'chMDB' heads(x, ..., n = 6L) ## S3 method for class 'fileMDB' heads(x, ..., n = 6L) heads(x, ..., n = 6L) ## S3 method for class 'memoMDB' heads(x, ..., n = 6L) ## S3 method for class 'metaMDB' heads(x, ..., n = 6L)## S3 method for class 'chMDB' heads(x, ..., n = 6L) ## S3 method for class 'fileMDB' heads(x, ..., n = 6L) heads(x, ..., n = 6L) ## S3 method for class 'memoMDB' heads(x, ..., n = 6L) ## S3 method for class 'metaMDB' heads(x, ..., n = 6L)
x |
an object with embedded data tables |
... |
the name of the tables to get (default: all of them) |
n |
maximum number of records to return (default: 6) |
A list of dplyr::tibble and matrix
Import a function to map collection members
import_collection_mapper(collection, fun)import_collection_mapper(collection, fun)
collection |
the name of the targeted collection
(it should belong to local collections: see |
fun |
a function which takes 2 data.frames (x an y) with fields described in the collection definition and map the different elements. |
No return value, called for side effects. The function will be used to map collection members.
Import a the definition of a collection of concepts in the local environment
import_local_collection(txt, overwrite = FALSE)import_local_collection(txt, overwrite = FALSE)
txt |
a JSON string or file |
overwrite |
a single logical. If TRUE the collection is overwritten if it already exists (default: FALSE) |
No return value, called for side effects. The collection will be available and operations will be possible on its members.
The initialization can only be done locally (host="localhost")
init_chTKCat(x, instance, version, path, login, password, contact)init_chTKCat(x, instance, version, path, login, password, contact)
x |
a chTKCat object |
instance |
instance name of the database |
version |
version name of the database |
path |
path to ClickHouse folder |
login |
login of the primary administrator of the database |
password |
password for the primary administrator of the database |
contact |
contact information for the primary administrator of the database |
a chTKCat
Is a chMDB public
is_chMDB_public(x, mdb)is_chMDB_public(x, mdb)
x |
a chTKCat object |
mdb |
name of the modeled database |
A logical indicating if the chMDB is public or not.
Check if the chMDB object refers to the current instance of the MDB
is_current_chMDB(x)is_current_chMDB(x)
x |
a chMDB object |
A single logical: TRUE if x refers to the current instance of the MDB.
Check if the object is a chMDB object
is.chMDB(x)is.chMDB(x)
x |
any object |
A single logical: TRUE if x is a chMDB object
Check the object is a chTKCat object
is.chTKCat(x)is.chTKCat(x)
x |
any object |
A single logical: TRUE if x is a chTKCat object
Check if the object is a fileMDB object
is.fileMDB(x)is.fileMDB(x)
x |
any object |
A single logical: TRUE if x is an fileMDB object
Check if an object represents Knowledge Management Requirements (KMR)
is.KMR(x)is.KMR(x)
x |
an object |
TRUE if x is a KMR, FALSE if not
Check if the object is an MDB object
is.MDB(x)is.MDB(x)
x |
any object |
A single logical: TRUE if x is an MDB object.
Check if the object is a memoMDB object
is.memoMDB(x)is.memoMDB(x)
x |
any object |
A single logical: TRUE if x is an memoMDB object
Check if the object is a metaMDB object
is.metaMDB(x)is.metaMDB(x)
x |
any object |
A single logical: TRUE if x is an metaMDB object
Check if the object is a POK object
is.POK(x)is.POK(x)
x |
any object |
A single logical: TRUE if x is a POK object
Check the object is a TKCat object
is.TKCat(x)is.TKCat(x)
x |
any object |
A single logical: TRUE if x is a TKCat object
Join connected tables
join_mdb_tables( x, ..., type = c("left", "right", "inner", "full"), jtName = NA )join_mdb_tables( x, ..., type = c("left", "right", "inner", "full"), jtName = NA )
x |
an MDB object |
... |
at least 2 names of tables to join |
type |
the type of join among:
|
jtName |
the name of the joint. IF NA (default), the name is then the name is the first provided table name. |
A metaMDB corresponding to x with the joined tables replaced by the joint. If less than 2 table names are provided, the function returns the original x MDB.
List instance timestamps of an MDB in chTKCat
list_chMDB_timestamps(x, name)list_chMDB_timestamps(x, name)
x |
a chTKCat object |
name |
the name of the database |
A tibble with the instance of each table at each timestamp. The "current" attribute indicate the current timestamp instance. If there is no recorded timestamp, the function returns NULL.
List users of an MDB of a chTKCat object
list_chMDB_users(x, mdbs = NULL)list_chMDB_users(x, mdbs = NULL)
x |
a chTKCat object |
mdbs |
names of the modeled databases. If NULL (default), all the databases are considered. |
A tibble with 3 columns:
user: the user login
mdb: the name of the modeled database
admin: if the user is an admin of the MDB
List collections available in a chTKCat
list_chTKCat_collections(x, withJson = FALSE)list_chTKCat_collections(x, withJson = FALSE)
x |
a chTKCat object |
withJson |
if TRUE, returns the json strings of the collection (default: FALSE) |
A tibble with the title, the description and optionally the json definition of the collections
List chTKCat user
list_chTKCat_users(x)list_chTKCat_users(x)
x |
a chTKCat object |
A tibble with 3 columns:
login: user login
contact: user contact information
admin: if the user is an admin of the chTKCat object
List properties of a feature
list_feature_properties(kmr, feature)list_feature_properties(kmr, feature)
kmr |
a KMR object |
feature |
the name of the feature |
A dplyr::tibble with the description of feature properties
List local collections of concepts
list_local_collections(withJson = FALSE)list_local_collections(withJson = FALSE)
withJson |
if TRUE, returns the json strings of the collection (default: FALSE) |
A tibble with the title, the description and optionally the json definition of the collections
List available MDB
## S3 method for class 'TKCat' list_MDBs(x, withInfo = TRUE) ## S3 method for class 'chTKCat' list_MDBs(x, withInfo = TRUE) list_MDBs(x, withInfo = TRUE)## S3 method for class 'TKCat' list_MDBs(x, withInfo = TRUE) ## S3 method for class 'chTKCat' list_MDBs(x, withInfo = TRUE) list_MDBs(x, withInfo = TRUE)
x |
|
withInfo |
if TRUE (default), the function returns a table with db_info. If FALSE, it returns only MDB names. |
A tibble with information about the MDB available in a TKCat related object.
List possible units for a type of measurement
list_measurement_units(kmr, measurement)list_measurement_units(kmr, measurement)
kmr |
a KMR object |
measurement |
the type of measurement |
A dplyr::tibble with the description of supported units
List supported types of measurement
list_measurements(kmr)list_measurements(kmr)
kmr |
a KMR object |
A vector of character with measurement names
List of supported values for an integer or a character property
list_property_values(kmr, feature, property)list_property_values(kmr, feature, property)
kmr |
a KMR object |
feature |
the name of the feature |
property |
the name of the property |
A dplyr::tibble with value and their description (if available)
List the features provided by a set of tables
list_table_features(kmr, tables = NULL)list_table_features(kmr, tables = NULL)
kmr |
a KMR object |
tables |
the name of the tables. If NULL (default), all the features are listed. |
A dplyr::tibble with feature description and properties
List types of tables defined in a KMR object
list_table_types(kmr)list_table_types(kmr)
kmr |
a KMR object |
A dplyr::tibble with the names of table types, their descriptions and the related collections
List tables in a clickhouse database
## S3 method for class 'DBIConnection' list_tables(x, dbNames = NULL, ...) ## S3 method for class 'chTKCat' list_tables(x, dbNames = NULL, ...) list_tables(x, ...)## S3 method for class 'DBIConnection' list_tables(x, dbNames = NULL, ...) ## S3 method for class 'chTKCat' list_tables(x, dbNames = NULL, ...) list_tables(x, ...)
x |
an object with a clickhouse connection |
dbNames |
the name of databases to focus on (default NULL ==> all) |
... |
method specific parameters |
A tibble with at least the following columns:
database: the name of the database
name: the name of the table
total_rows: the number of rows in the table
total_bytes: the size of the table
total_columns: the number of columns in the table
Manage user information in a shiny interface
manage_chTKCat_users(x, pwdFile = NULL)manage_chTKCat_users(x, pwdFile = NULL)
x |
a chTKCat object |
pwdFile |
a local file in which the password for x can be found. If NULL (default), the connection is shared by all sessions and can be disabled at some point. |
Map different collection members
map_collection_members( x, y, collection, xm, ym, suffix = c("_x", "_y"), fun = NA, ... )map_collection_members( x, y, collection, xm, ym, suffix = c("_x", "_y"), fun = NA, ... )
x |
a data.frame |
y |
a data.frame |
collection |
the name of the collection. |
xm |
collection member x: a data.frame with the fields
"field", "static", "value", "type" as returned by
the |
ym |
collection member y: a data.frame with the fields
"field", "static", "value", "type" as returned by
the |
suffix |
the suffix to append to field names from x and y tables.
Default: |
fun |
the function used to map x and y collection members. By default (NA) it is automatically identified if recorded in the system. The way to write this function is provided in the details section. |
... |
additional parameters for the fun function. |
fun must have at least an x and a y parameters.
Each of them should be a data.frame with all the field values
given in xm and ym. Additional parameters
can be defined and will be forwarded using ....
fun should return a data frame with all the fields values
given in xm and ym followed by "_x" and "_y" suffix.
A tibble giving necessary information to map elements in x and y.
The columns corresponds to the field values in xm and ym followed by a
suffix (default: c("_x", "_y")). Only fields documented as non static
in xm and ym are kept.
The class "MDB" provides general functions for handling modeled databases. The MDB classes implemented in the TKCat package are: fileMDB, memoMDB, chMDB and metaMDB. These classes provide additional functions.
## S3 method for class 'MDB' names(x) ## S3 method for class 'MDB' length(x) ## S3 method for class 'MDB' lengths(x, use.names = TRUE) ## S3 method for class 'MDB' as.list(x, ...) ## S3 method for class 'MDB' select(.data, ...) ## S3 method for class 'MDB' pull(.data, var = -1, name = NULL, ...) ## S3 method for class 'MDB' c(...) ## S3 method for class 'MDB' merge( x, y, by = get_shared_collections(x, y), dbInfo = list(name = paste(db_info(x)$name, db_info(y)$name, sep = "_")), dmAutoLayout = TRUE, rtColor = "yellow", funs = list(), ... )## S3 method for class 'MDB' names(x) ## S3 method for class 'MDB' length(x) ## S3 method for class 'MDB' lengths(x, use.names = TRUE) ## S3 method for class 'MDB' as.list(x, ...) ## S3 method for class 'MDB' select(.data, ...) ## S3 method for class 'MDB' pull(.data, var = -1, name = NULL, ...) ## S3 method for class 'MDB' c(...) ## S3 method for class 'MDB' merge( x, y, by = get_shared_collections(x, y), dbInfo = list(name = paste(db_info(x)$name, db_info(y)$name, sep = "_")), dmAutoLayout = TRUE, rtColor = "yellow", funs = list(), ... )
x |
an MDB object |
use.names |
return the names of the tables |
... |
additional parameters |
.data |
an MDB object |
var |
a variable specified as in dplyr::pull |
name |
not used but kept for compatibility with the generic function |
y |
an MDB object |
by |
a tibble as returned by the |
dbInfo |
a list with DB information: "name" (only mandatory field), "title", "description", "url", "version", "maintainer". |
dmAutoLayout |
if TRUE (default) the layout of the merged data model is automatically adjusted. |
rtColor |
the color of the relational tables in the merged data model (default: "yellow") |
funs |
a named list of functions (default: list()). If there is
no function for mapping a collection in this list, it is taken
automatically using the |
names() returns the table names.
length() returns the number of tables in x.
lengths() returns the number of fields for each table in x.
as.list.MDB() returns a simple list of tibbles with all the
data from the tables in x.
A metaMDB object gathering x and y along
with relational tables between them created using collection members
and mapping functions automatically chosen or provided by
the funs parameter. ... can be used to send parameters to the mapper
functions.
MDB methods: db_info, data_model, data_tables, collection_members, count_records, filter_with_tables, as_fileMDB Additional documentation is provided for each specific class: fileMDB, memoMDB, chMDB and metaMDB.
Get a list of MDB from metaMDB object
MDBs(x)MDBs(x)
x |
a metaMDB object |
A list of MDB objects
An MDB (Modeled DataBase) in memory: memoMDB
Rename tables of a memoMDB object
memoMDB( dataTables, dataModel, dbInfo, collectionMembers = NULL, check = TRUE, checks = c("unique", "not nullable", "foreign keys"), verbose = FALSE ) ## S3 replacement method for class 'memoMDB' names(x) <- value ## S3 method for class 'memoMDB' rename(.data, ...) ## S3 method for class 'memoMDB' x[i] ## S3 method for class 'memoMDB' x[[i]] ## S3 method for class 'memoMDB' x$imemoMDB( dataTables, dataModel, dbInfo, collectionMembers = NULL, check = TRUE, checks = c("unique", "not nullable", "foreign keys"), verbose = FALSE ) ## S3 replacement method for class 'memoMDB' names(x) <- value ## S3 method for class 'memoMDB' rename(.data, ...) ## S3 method for class 'memoMDB' x[i] ## S3 method for class 'memoMDB' x[[i]] ## S3 method for class 'memoMDB' x$i
dataTables |
a list of tibbles |
dataModel |
a ReDaMoR::RelDataModel object |
dbInfo |
a list with DB information: "name" (only mandatory field), "title", "description", "url", "version", "maintainer". |
collectionMembers |
the members of collections as provided to the collection_members<- function (default: NULL ==> no member). |
check |
logical: if TRUE (default) the data are confronted to the data model |
checks |
a character vector with the name of optional checks to be done (all of them c("unique", "not nullable", "foreign keys")) |
verbose |
if TRUE display the data confrontation report (default: FALSE) |
x |
a memoMDB object |
value |
new table names |
.data |
a memoMDB object |
... |
Use new_name = old_name to rename selected tables |
i |
the index or the name of the tables to take |
A memoMDB object
MDB methods: db_info, data_model, data_tables, collection_members, count_records, dims, filter_with_tables, as_fileMDB
Additional general documentation is related to MDB.
hpo <- read_fileMDB( path=system.file("examples/HPO-subset", package="ReDaMoR"), dataModel=system.file("examples/HPO-model.json", package="ReDaMoR"), dbInfo=list( "name"="HPO", "title"="Data extracted from the HPO database", "description"=paste( "This is a very small subset of the HPO!", "Visit the reference URL for more information" ), "url"="http://human-phenotype-ontology.github.io/" ) ) %>% as_memoMDB() count_records(hpo) ## Too long on win-builder.r-project.org ## Not run: hpoSlice <- slice(hpo, HPO_diseases=1:10) count_records(hpoSlice) if("stringr" %in% installed.packages()[,"Package"]){ epilHP <- filter( hpo, HPO_diseases=stringr::str_detect( label, stringr::regex("epilepsy", ignore_case=TRUE) ) ) count_records(epilHP) } ## End(Not run)hpo <- read_fileMDB( path=system.file("examples/HPO-subset", package="ReDaMoR"), dataModel=system.file("examples/HPO-model.json", package="ReDaMoR"), dbInfo=list( "name"="HPO", "title"="Data extracted from the HPO database", "description"=paste( "This is a very small subset of the HPO!", "Visit the reference URL for more information" ), "url"="http://human-phenotype-ontology.github.io/" ) ) %>% as_memoMDB() count_records(hpo) ## Too long on win-builder.r-project.org ## Not run: hpoSlice <- slice(hpo, HPO_diseases=1:10) count_records(hpoSlice) if("stringr" %in% installed.packages()[,"Package"]){ epilHP <- filter( hpo, HPO_diseases=stringr::str_detect( label, stringr::regex("epilepsy", ignore_case=TRUE) ) ) count_records(epilHP) } ## End(Not run)
Create a ClickHouse MergeTree table from a ReDaMoR::RelTableModel
mergeTree_from_RelTableModel(con, dbName, tm)mergeTree_from_RelTableModel(con, dbName, tm)
con |
the clickhouse connection |
dbName |
the name of the database in which the table should be written |
tm |
a ReDaMoR::RelTableModel object |
No return value, called for side effects
Create ClickHouse MergeTree tables from a ReDaMoR::RelDataModel
mergeTrees_from_RelDataModel(con, dbName, dbm)mergeTrees_from_RelDataModel(con, dbName, dbm)
con |
the clickhouse connection |
dbName |
the name of the database in which the tables should be written |
dbm |
a ReDaMoR::RelDataModel object |
No return value, called for side effects
A metaMDB object is an MDB gathering several other MDBs glued by relational tables.
metaMDB(MDBs, relationalTables, dataModel, dbInfo, check = TRUE) ## S3 replacement method for class 'metaMDB' names(x) <- value ## S3 method for class 'metaMDB' rename(.data, ...) ## S3 method for class 'metaMDB' x[i] ## S3 method for class 'metaMDB' x[[i]] ## S3 method for class 'metaMDB' x$imetaMDB(MDBs, relationalTables, dataModel, dbInfo, check = TRUE) ## S3 replacement method for class 'metaMDB' names(x) <- value ## S3 method for class 'metaMDB' rename(.data, ...) ## S3 method for class 'metaMDB' x[i] ## S3 method for class 'metaMDB' x[[i]] ## S3 method for class 'metaMDB' x$i
MDBs |
a list of MDB objects |
relationalTables |
a list of tibbles corresponding to the relational tables between the different MDBs |
dataModel |
a ReDaMoR::RelDataModel object gathering all the data model of all the MDBs plus the relational tables |
dbInfo |
a list with DB information: "name" (only mandatory field), "title", "description", "url", "version", "maintainer". |
check |
logical: if TRUE (default) the data are confronted to the data model |
x |
a metaMDB object |
value |
new table names |
.data |
a metaMDB object |
... |
Use new_name = old_name to rename selected tables |
i |
the index or the name of the tables to take |
A metaMDB object
MDB methods: db_info, data_model, data_tables, collection_members, count_records, dims, filter_with_tables, as_fileMDB
Additional general documentation is related to MDB.
get_confrontation_report, ReDaMoR::format_confrontation_report and ReDaMoR::format_confrontation_report_md for getting and formatting the report confronting the data to the model.
Parse source code to get R helpers
parse_R_helpers(code, ...)parse_R_helpers(code, ...)
code |
the source code as a character vector |
... |
other objects to add in the environment of the returned functions |
Functions in code must be documented with roxygen2::roxygen2-package tags and only functions with the '@export' tag are returned.
A list of functions from code plus an "help" function used to get
function documentation.
{ code <- " a <- 2 #' Set the 'a' value to use in the add_a function #' #' @param v a numeric value #' #' @return Does not return anything #' #' @export set_a <- function(v) a <<- v #' Add a 'a' value defined separately #' #' @param x a numeric value #' #' @return x + a #' #' @export add_a <- function(x) x + a #' Add a 'a' value defined separately to a b value made available #' in environment #' #' #' @return b + a #' #' @export add_a_to_b <- function() b + a " helpers <- parse_R_helpers(code, b=3) helpers$help() helpers$help("add_a") helpers$add_a(3.5) helpers$set_a(4) helpers$add_a(3.5) helpers$add_a_to_b() helpers <- parse_R_helpers(code, b=6) helpers$add_a_to_b() }{ code <- " a <- 2 #' Set the 'a' value to use in the add_a function #' #' @param v a numeric value #' #' @return Does not return anything #' #' @export set_a <- function(v) a <<- v #' Add a 'a' value defined separately #' #' @param x a numeric value #' #' @return x + a #' #' @export add_a <- function(x) x + a #' Add a 'a' value defined separately to a b value made available #' in environment #' #' #' @return b + a #' #' @export add_a_to_b <- function() b + a " helpers <- parse_R_helpers(code, b=3) helpers$help() helpers$help("add_a") helpers$add_a(3.5) helpers$set_a(4) helpers$add_a(3.5) helpers$add_a_to_b() helpers <- parse_R_helpers(code, b=6) helpers$add_a_to_b() }
Read a collection member JSON file
read_collection_members(txt)read_collection_members(txt)
txt |
a JSON string or file |
A tibble with the description of the collection members of a resource
Read a fileMDB from a path
read_fileMDB( path, dbInfo = NULL, dataModel = NULL, collectionMembers = NULL, check = TRUE, n_max = 10, verbose = TRUE )read_fileMDB( path, dbInfo = NULL, dataModel = NULL, collectionMembers = NULL, check = TRUE, n_max = 10, verbose = TRUE )
path |
the path to a folder with data or with the following structure:
|
dbInfo |
a list or a json file with DB information:
"name" (only mandatory field), "title", "description",
"url" (or "reference URL"),
"version", "maintainer". If NULL (default), the DESCRIPTION.json file found
in path. This file should also contains relevant parameters for the
|
dataModel |
a ReDaMoR::RelDataModel object or json file. If NULL (default), the model json file found in path/model. |
collectionMembers |
the members of collections as provided to the collection_members<- function. If NULL (default), the members are taken from json files found in path/model/Collections |
check |
logical: if TRUE (default) the data are confronted to the data model |
n_max |
maximum number of records to read
for checks purpose (default: 10). See also |
verbose |
if TRUE (default) display the data confrontation report |
A fileMDB object
get_confrontation_report, ReDaMoR::format_confrontation_report and ReDaMoR::format_confrontation_report_md for getting and formatting the report confronting the data to the model.
Read KMR from a path
read_KMR(...)read_KMR(...)
... |
parameters for |
A KMR object
Get a list of relational tables
relational_tables(x, recursive = FALSE)relational_tables(x, recursive = FALSE)
x |
a metaMDB object |
recursive |
if TRUE, function returns also the relational tables from embedded metaMDBs. |
A list of relational tables (tibbles)
Drop a user of an MDB of a chTKCat object
remove_chMDB_user(x, mdb, login)remove_chMDB_user(x, mdb, login)
x |
a chTKCat object |
mdb |
name of the modeled database |
login |
login of the user to drop |
No return value, called for side effects
Remove a collection from a chTKCat database
remove_chTKCat_collection(x, title)remove_chTKCat_collection(x, title)
x |
a chTKCat object |
title |
the title of the collection to remove |
No return value, called for side effects
Remove KM feature specifications from an MDB object
rm_km_feature(x, kmr, table, feature)rm_km_feature(x, kmr, table, feature)
x |
an MDB object to update with specification tables |
kmr |
an MDB object with KM requirements |
table |
the name of an existing table in x |
feature |
the name of a feature with specification in x table |
An MDB object with relevant KM table feature specification removed
Remove KM table specifications from an MDB object
rm_km_table(x, kmr, table)rm_km_table(x, kmr, table)
x |
an MDB object to update with specification tables |
kmr |
an MDB object with KM requirements |
table |
the name of an existing table in x specification |
An MDB object with relevant KM table specification removed
Scan a catalog of fileMDB
scan_fileMDBs(path, subdirs = NULL, check = TRUE, n_max = 10)scan_fileMDBs(path, subdirs = NULL, check = TRUE, n_max = 10)
path |
directory from which all the fileMDB should be read |
subdirs |
the sub directories (relative to path) to take into account. If NULL (default) all the sub directories are considered. |
check |
logical: if TRUE (default) the data are confronted to the data model |
n_max |
maximum number of records to read
for checks purpose (default: 10). See also |
a TKCat object
Search fields in a TKCat related object
## S3 method for class 'TKCat' search_MDB_fields(x, searchTerm) ## S3 method for class 'chTKCat' search_MDB_fields(x, searchTerm) search_MDB_fields(x, searchTerm)## S3 method for class 'TKCat' search_MDB_fields(x, searchTerm) ## S3 method for class 'chTKCat' search_MDB_fields(x, searchTerm) search_MDB_fields(x, searchTerm)
x |
|
searchTerm |
a single character with the term to search |
An MDB object
Search tables in a TKCat related object
## S3 method for class 'TKCat' search_MDB_tables(x, searchTerm) ## S3 method for class 'chTKCat' search_MDB_tables(x, searchTerm) search_MDB_tables(x, searchTerm)## S3 method for class 'TKCat' search_MDB_tables(x, searchTerm) ## S3 method for class 'chTKCat' search_MDB_tables(x, searchTerm) search_MDB_tables(x, searchTerm)
x |
|
searchTerm |
a single character with the term to search |
An MDB object
Set chMDB access
set_chMDB_access(x, mdb, public)set_chMDB_access(x, mdb, public)
x |
a chTKCat object |
mdb |
name of the modeled database |
public |
if access is public |
No return value, called for side effects
Set timestamp of the current version of an MDB in chTKCat
set_chMDB_timestamp(x, name, timestamp)set_chMDB_timestamp(x, name, timestamp)
x |
a chTKCat object |
name |
the name of the database to affect |
timestamp |
a single POSIXct value as a timestamp for the chMDB instance. |
No return value, called for side effects
This function prints details regarding a collection: title, description
and arguments information. These arguments are those that can be
used to document collection members within an MDB using
the add_collection_member() function.
show_collection_def(collection, silent = FALSE)show_collection_def(collection, silent = FALSE)
collection |
a json string with the collection definition as returned
by |
silent |
a logical indicating if the definition should be written (TRUE by default) or not. |
A list with:
collection title
collection description
a list of arguments for defining collection members as a list of elements with:
the type of the argument element
allowed values if any
get_local_collection("BE") %>% show_collection_def()get_local_collection("BE") %>% show_collection_def()
Subset a chMDB object according to row position in one table and return a memoMDB
## S3 method for class 'chMDB' slice(.data, ..., by = 10^5, .preserve = FALSE)## S3 method for class 'chMDB' slice(.data, ..., by = 10^5, .preserve = FALSE)
.data |
a chMDB object |
... |
a single argument. The name of this argument should be a table name of x and the value of this argument should be vector of integers corresponding to row indexes. |
by |
the size of the batch: number of records to slice together (default: 10^5) |
.preserve |
not used |
a memoMDB object
Subset a fileMDB object according to row position in one table and return a memoMDB
## S3 method for class 'fileMDB' slice(.data, ..., .preserve = FALSE)## S3 method for class 'fileMDB' slice(.data, ..., .preserve = FALSE)
.data |
a fileMDB object |
... |
a single argument. The name of this argument should be a table name of x and the value of this argument should be vector of integers corresponding to row indexes. |
.preserve |
not used |
a memoMDB object
Subset a memoMDB object according to row position in one table
## S3 method for class 'memoMDB' slice(.data, ..., .preserve = FALSE)## S3 method for class 'memoMDB' slice(.data, ..., .preserve = FALSE)
.data |
a memoMDB object |
... |
a single argument. The name of this argument should be a table name of x and the value of this argument should be vector of integers corresponding to row indexes. |
.preserve |
not used |
a memoMDB object
Subset a metaMDB object according to row position in one table
## S3 method for class 'metaMDB' slice(.data, ..., .preserve = FALSE)## S3 method for class 'metaMDB' slice(.data, ..., .preserve = FALSE)
.data |
a metaMDB object |
... |
a single argument. The name of this argument should be a table name of x and the value of this argument should be vector of integers corresponding to row indexes. |
.preserve |
not used |
a memoMDB object
TKCat: a catalog of MDB
Rename a TKCat object
TKCat(..., list = NULL) ## S3 replacement method for class 'TKCat' names(x) <- value ## S3 method for class 'TKCat' rename(.data, ...) ## S3 method for class 'TKCat' x[i] ## S3 method for class 'TKCat' c(...)TKCat(..., list = NULL) ## S3 replacement method for class 'TKCat' names(x) <- value ## S3 method for class 'TKCat' rename(.data, ...) ## S3 method for class 'TKCat' x[i] ## S3 method for class 'TKCat' c(...)
... |
TKCat objects |
list |
a list of MDB objects |
x |
a TKCat object |
value |
new MDB names |
.data |
a TKCat object |
i |
index or names of the MDB to take |
a TKCat object
Unarchive a chMDB in a chTKCat
unarchive_chMDB(x, name)unarchive_chMDB(x, name)
x |
a chTKCat object |
name |
the name of the database to archive |
No return value, called for side effects
The update is done automatically based on user access.
update_chMDB_grants(x, mdb)update_chMDB_grants(x, mdb)
x |
a chTKCat object |
mdb |
name of the modeled database |
No return value, called for side effects
Update a chTKCat user information
update_chTKCat_user(x, login, contact, admin, provider)update_chTKCat_user(x, login, contact, admin, provider)
x |
a chTKCat object |
login |
user login |
contact |
contact information (can be NA) |
admin |
a logical indicating if the user is an admin of the chTKCat instance |
provider |
a logical indicating if the user is data provider (TRUE) or a data consumer (FALSE: default) |
No return value, called for side effects
Write a collection member JSON file
write_collection_members(colMembers, path = NA, collection = NULL)write_collection_members(colMembers, path = NA, collection = NULL)
colMembers |
A tibble as returned by |
path |
the JSON file to write. If |
collection |
The collection definition (json string).
If NULL (default), it is taken from TKCat environment
(see |
The JSON representation of collection members. If a path is provided, then the JSON is also written in it.
Write a Clickhouse MergeTree table
write_MergeTree( con, dbName, tableName, value, rtypes = NULL, nullable = NULL, sortKey = NULL )write_MergeTree( con, dbName, tableName, value, rtypes = NULL, nullable = NULL, sortKey = NULL )
con |
the clickhouse connection |
dbName |
the name of the database |
tableName |
the name of the table |
value |
the table to import |
rtypes |
a named character vector giving the R type of each and every columns. If NULL (default), types are guessed from value. |
nullable |
a character vector indicating the name of the columns which are nullable (default: NULL) |
sortKey |
a character vector indicating the name of the columns used in the sort key. If NULL (default), all the non-nullable columns are used in the key. |
No return value, called for side effects