I have been an avid user and advocate of open source since the early days of Linux (circa 1995). I also make frequent contributions to open source and authored a few of my own.

github.com/sujee   •   github.com/elephantscale

My Open Source Projects

Allycat

Allycat is an end to end open-source RAG pipeline for website content. It can

  • scape websites
  • clean up content
  • index content, vectorize and store them in a vector database
  • and a UI for queries.

The entire stack is open source. It supports LLMs running locally or using an inference service.

repo

MongoDB as Vector Database

Vector datastores are crucial for developing RAG applications with LLMs.

This resppostirory features code and reference architectures for vector search and RAG applications with MongoDB.

I have experimented with various embedding models and LLMs (openAI, Mistral, LLama)

code

Dockerized Stacks

I created these dockerized stacks to make development and running them easier

Hadoop DNS Checker

Hadoop is very particular about DNS records of servers in the cluster. DNS record mis matches can cause runtime errors.

My hadoop DNS checker utility verifies DNS records of cluster machines.

My Open source contributions

Spark Job Server

Spark Job Server allows running Spark jobs with low latency.

Submitted multiple patches and pull requests

HBase

Contributed performance patch and document patches to HBase - a distributed noSQL database

  • HBASE-4440 : A write benchmark writes lot of records. Then when a region splits, the writes are paused until the region is split and migrated to another server. This delay negatively affects the benchmark. My patch adds an option to pre-split the table, so the writes can be performed in parallel acros multiple regions / servers
  • HBASE-5555 - documentation and scripts to verify DNS records of HBase machines.

More

more