Tailored Knowledge Catalog

The aim of TKCat (Tailored Knowledge Catalog) is to facilitate the management of data from knowledge resources that are frequently used alone or together in research environments. In TKCat, knowledge resources are manipulated as modeled database (MDB) objects. These objects provide access to the data tables along with a general description of the resource and a detailed data model generated with ReDaMoR documenting the tables, their fields and their relationships. These MDB are then gathered in catalogs that can be easily explored and shared. TKCat provides tools to easily subset, filter and combine MDBs and create new catalogs suited for specific needs.

This package has been presented at the useR!2022 conference and here are the slides.

The TKCat R package is licensed under GPL-3.

Installation

From CRAN

install.packages("TKCat")

Dependencies

The following R packages available on CRAN are required:

• ReDaMoR: Relational Data Modeler
• DBI: R Database Interface
• visNetwork: Network Visualization using ‘vis.js’ Library
• dplyr: A Grammar of Data Manipulation
• ClickHouseHTTP: A Simple HTTP Database Interface to ‘ClickHouse’
• rlang: Functions for Base Types and Core R and ‘Tidyverse’ Features
• tidyselect: Select from a Set of Strings
• askpass: Password Entry Utilities for R, Git, and SSH
• shiny: Web Application Framework for R
• shinydashboard: Create Dashboards with ‘Shiny’
• DT: A Wrapper of the JavaScript Library ‘DataTables’
• htmltools: Tools for HTML
• httpuv: HTTP and WebSocket Server Library
• readr: Read Rectangular Text Data
• jsonlite: A Simple and Robust JSON Parser and Generator for R
• jsonvalidate: Validate ‘JSON’ Schema
• markdown: Render Markdown with ‘commonmark’
• promises: Abstractions for Promise-Based Asynchronous Programming
• future: Unified Parallel and Distributed Processing in R for Everyone
• xml2: Parse XML
• Matrix: Sparse and Dense Matrix Classes and Methods
• uuid: Tools for Generating and Handling of UUIDs
• crayon: Colored Terminal Output
• roxygen2: In-Line Documentation for R

And those are suggested:

• knitr: A General-Purpose Package for Dynamic Report Generation in R
• rmarkdown: Dynamic Documents for R
• RClickhouse: ‘Yandex Clickhouse’ Interface for R with Basic ‘dplyr’ Support
• stringr: Simple, Consistent Wrappers for Common String Operations
• data.tree: General Purpose Hierarchical Data Structure
• BED: Biological Entity Dictionary (BED)

From github

pak::pkg_install("github::patzaw/TKCat")

Documentation

• Introduction to TKCat (vignette("TKCat")): the main guide. It shows how to create MDBs (data tables, description and data model), how to explore, filter, join and combine them, and how to manage them in local catalogs or in a centralized ClickHouse catalog (chTKCat).

• TKCat end-to-end (vignette("TKCat-EndToEnd")): a step-by-step walkthrough of a complete deployment: setting up ClickHouse, preparing a few MDBs and pushing them to the database, querying them, and configuring an MCP server and its clients.

• Querying a chTKCat instance with an AI assistant (MCP server) (vignette("TKCat-MCP")): how to run the TKCat MCP server, locally (stdio) or as a remote service (HTTP), so that AI assistants such as Claude can explore and query the MDBs of a chTKCat instance. It also covers the available tools and security considerations.

• Defining and using Requirements for Knowledge Management (vignette("TKCat-KMR-POK")): how to define shared requirements (KMR) that MDBs can comply with, how to refer to them in MDBs, and how to use KMR and MDBs together to retrieve pieces of knowledge (POK) in a consistent way.

Alternatives

• The dm package provides similar features but with different implementation choices. Here are the main differences:

  • The dm data model feature is built upon the datamodelr package whereas TKCat relies on ReDaMoR.
  • Both dm and TKCat provides mechanisms to check the fulfillment of the data model constraints and tools to automatically take advantage of them.
  • dm supports connection to many different DBMS. It also takes advantage of constraints which are documented in the DBMS when available. TKCat only supports the ClickHouse system through the ClickHouseHTTP or RClickhouse packages.
  • TKCat implements 3 main types of MDB based on files, memory tables or ClickHouse database. It also provides mechanisms to automatically convert from and to any of these implementations.
  • TKCat supports catalogs of MDBs facilitating the exploration of existing data. It also allows the integration of different MDBs through the automatic identification of similar concepts (Collections) and the automatic conversion of the different vocabulary on which they rely.