Build a modern LLM from scratch. Every line commented. Explained like we are five.
Ungreedy subword tokenizer and vocabulary trainer for Python, Go, C++ & Javascript
TokenScript schema, specs and paper
Frames iOS: making native card payments simple
Open morphology for Finnish
Taiwanese Hokkien Transliterator and Tokeniser
A Python 3 module that provides functions for splitting identifiers found in source code files.
Built a complete search engine by creating an Inverted Index on the Wikipedia corpus ( of 2018 with size 72 GB). That gives you top search result related to given query words.
Taiwanese Hokkien Transliterator and Tokeniser
Frames Android: making native card payments simple
Reference implementation of a securities register for tokenised private-market debt. The off-chain register is authoritative; the chain is a derived mirror.
Chatbot pédagogique entraîné sur un large fichier d’intents (thèmes d’éducation/psycho/socio/éco). Tokenisation, réseau LSTM Keras, entraînement prolongé et boucle d’inférence simple pour répondre aux intentions.
eqCAT — Tokenised Parametric Earthquake Catastrophe Bond | ERC-20 smart contract enabling fractional on-chain access to the $64B insurance-linked securities market | Solidity 0.8.24 • OpenZeppelin • Hardhat
Obsidian vault on the tokenisation of real-world assets - finance, law, carbon credits, real estate, property rights, stablecoins, agentic AI
HIPAA-regulated AI integration architecture for Impowr's healthcare platform, serving at-risk populations. Deterministic tokenization on Azure (SQL, Blob, OpenAI) for secure ingestion of sensitive health data; NLP post-processing translates AI outputs into clinical insights for non-technical staff.
This project consist of creating a system or tool that can understand and transform a textual description of a process into an image.
An objective diagnostic framework that measures digital maturity
Exploratory data analysis and Natural Language Processing techniques applied to a corpus of emails to identify the key topics and their frequencies
To associate your repository with the tokenisation topic, visit your repo's landing page and select "manage topics."