2024

Im2Latex: Image-to-LaTeX Converter for Mathematical Formulas and Text

In this project, we train a vision encoder-decoder model to generate LaTeX code from images of mathematical formulas and text. Utilizing a diverse collection of image-to-LaTeX data, we build two models: a base model with a Swin Transformer encoder and a GPT-2 decoder, trained on machine-generated images, and a fine-tuned version enhanced with Low-Rank Adaptation (LoRA) trained on handwritten formulas. We then compare the BLEU performance of our specialized model on a handwritten test set with other similar models, such as Pix2Text, TexTeller, and Sumen. Through this project, we contribute open-source models for converting images to LaTeX and provide from-scratch code for building these models with distributed training and GPU optimizations. Our paper-report is available on Arxiv

2024

Injecting Graph Knowledge into LLMs for LRLs

This project explores the integration of multilingual graph knowledge from linguistic ontologies into multilingual Large Language Models using adapters to enhance performance for low-resource languages. Inspired by recent works such as K-ADAPTER and MAD-X, we propose a similar approach for incorporating knowledge from ConceptNet into multilingual LLMs for LRLs. We presented the paper describing our methodology and experimental results at the 1st Workshop on Knowledge Graphs and Large Language Models (KaLLM), co-located with ACL 2024. Our paper is available here

2024

Multilingual ConceptNet Embeddings

This application focuses on extracting data from ConceptNet for a specific language, saving the data into a JSON file, building PPMI embeddings from this data, and retrofitting additional GloVe embeddings (if provided) using the ConceptNet PPMI embeddings.

2023

Mental Health in Literature Analysis

In this project, we tried to diagnose literary characters with mental disorders according to International Classification of Diseases (ICD). Try out our HuggingFace Spaces based mini-app where you can diagnose any other literary character, or even yourself, based on provided description (please, note this can't be used for diagnostics and can be used for entertainment only).

2023

Vector Space Retrieval Model

This program is a search engine that searches for relevant documents in a given collection based on a query. The search engine uses the vector space retrieval model for specifying how to rank the documents. The collection of documents is stored in an XML file, which is preprocessed and indexed using TF-IDF scores. The core of the search engine is an index that weights terms according to the tf.idf weighting scheme.

2022

Mensa Telegram Bot

This project is a Python script that utilizes web scraping and the Telegram Bot API to provide translated cafeteria menus. It uses BeautifulSoup for web scraping, the googletrans library for translation, and telegram.ext for creating a Telegram bot.