[ECCV 2024] Official PyTorch implementation of RoPE-ViT "Rotary Position Embedding for Vision Transformer"
-
Updated
Oct 29, 2025 - Python
#
rotary-position-embedding
Here are 18 public repositories matching this topic...
[ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Models
-
Updated
Mar 12, 2024 - Python
[ECCV 2026] Official implementation of "MemRoPE: Training-Free Infinite Video Generation via Evolving Memory Tokens"
rope dit video-generation diffusion-models kv-cache long-video-generation rotary-position-embedding diffusion-transformer video-diffusion eccv2026 autoregressive-video
-
Updated
Jun 24, 2026 - Python
One Diffusion model implementation base on LibTorch
pytorch diffusion rotary libtorch diffusion-models ddpm ddim rotary-position-embedding rotary-2d-position-embedding rotary-position-encoding rotary-2d-position-encoding rectified-flow
-
Updated
Mar 22, 2023 - C++
CoPE: Clipped RoPE as A Scalable Free Lunch for Long Context LLMs
-
Updated
May 2, 2026 - Python
PyTorch implementation of Rotary Spatial Embeddings
pytorch transformer attention positional-encoding rotary-position-embedding rotary-position-encoding
-
Updated
Nov 13, 2025 - Python
Decoder-only transformer for molecule generation (SMILES representation).
-
Updated
Dec 27, 2024 - Jupyter Notebook
Image Captioning With MobileNet-LLaMA 3
nlp cnn pytorch transformer image-captioning image-text flickr8k-dataset mobilenetv3 pytorch-lightning kv-cache rotary-position-embedding grouped-query-attention rms-norm llama3
-
Updated
Jun 23, 2024 - Jupyter Notebook
This repository contains a lightweight PyTorch implementation of Qwen 3-style transformer components in qwen3_from_scratch.py.
-
Updated
May 6, 2026 - Python
面向 Transformer 推理的可验证 Triton 融合算子与 torch.library 集成
python acceleration deep-learning high-performance cuda inference pytorch nvidia stable triton gpu-computing gpu-kernels kernel-fusion ai-infra llm rotary-position-embedding rmsnorm llm-inference operator-fusion transformer-inference
-
Updated
Aug 31, 2026 - Python
A clean, efficient implementation of Rotary Positional Embeddings (RoPE) for transformers, with support for advanced variants like YARN scaling.
-
Updated
Apr 2, 2026 - Python
Transformer Models for Humorous Text Generation. Fine-tuned on Russian jokes dataset with ALiBi, RoPE, GQA, and SwiGLU.Plus a custom Byte-level BPE tokenizer.
nlp pytorch alibi transformer-models rotary-position-embedding grouped-query-attention swiglu bpe-tokenizer
-
Updated
Mar 10, 2025 - Jupyter Notebook
Machine Learning from scratch in C
embeddings edit-distance transformer neural-networks pca beam-search svd approximate-nearest-neighbor-search qr-decomposition softmax adam-optimizer ctc-loss relu lstm-neural-network multihead-attention svd-factorization gelu sigmoid-activation rotary-position-embedding machine-learning-in-c
-
Updated
Sep 5, 2026 - C
Decoder-only LLM trained on the Harry Potter books.
transformer byte-pair-encoding rotary-position-embedding large-language-model flash-attention grouped-query-attention
-
Updated
Dec 20, 2024 - Python
python deep-learning cuda transformers pytorch embeddings triton rope gpu-kernels alibi positional-encoding vision-transformer llm rotary-position-embedding fused-kernels
-
Updated
Jul 13, 2026 - Python
minimal Implementation of qwen 3 from scratch including both dense and moe model architectures
-
Updated
Aug 27, 2026 - Python
RootMeanSquareNorm + Rotary Position Embedding - Triton Kernel Optimization
pytorch rope rotary-position-embedding rmsnorm openai-triton triton-language triton-kernels throughput-optimization
-
Updated
Aug 3, 2026 - Python
A from-scratch implementation of a T5 model modified with Rotary Position Embeddings (RoPE). This project includes the code for pre-training on the C4 dataset in streaming mode with Flash Attention 2.
nlp pytorch sequence-to-sequence language-model from-scratch rope pre-training huggingface t5 evaluation-benchmark llm rotary-position-embedding flash-attention c4-dataset span-corruption
-
Updated
Jul 9, 2025 - Python
Add this topic to your repo
To associate your repository with the rotary-position-embedding topic, visit your repo's landing page and select "manage topics."