Elemental ODM

Home

Domain

Milestones

Documents

Presentations

About Us

Welcome to Elemental

A modern MongoDB ODM for Golang.

Try now!

Domain

Literature Review

Go was designed to solve challenges such as slow build times and uncontrolled dependencies that arise in large-scale software systems, making it particularly suitable for high-performance applications and distributed systems [1]. Therefore, an effective Object Relational Mapper is essential for simplifying the interaction between Go applications and NoSQL databases. Current tools like MGM and Mongo-Driver already provide useful abstractions for NoSQL databases, but each has its limitations [2]. These limitations make it critical to develop a more comprehensive NoSQL ORM tailored to Go, which can handle complex data models and relationships while offering cross-database compatibility and optimized performance. The simplicity of Go’s syntax, along with its built-in concurrency support and cross-platform compatibility, enables the efficient mapping of relational data to Go’s struct types while keeping the system performant. This efficiency is crucial for database-driven applications, which require frequent, optimized database interactions [3]. Therefore, creating an ORM for Go would help developers manage database interactions more seamlessly while leveraging the language’s strengths in scalability and performance [1].

References

[1] J. K. B, N. Shivraj, N. Rakshith, and N. M, “Study on go programming language,” International Journal of Advanced Research in Science Communication and Technology, pp. 330–333, Nov. 2021, doi: 10.48175/ijarsct-2126.
[2] Tramposo, “Database Interactions in Go: From SQL to NoSQL,” DEV Community, Oct. 01, 2024. https://dev.to/tramposo/database-interactions-in-go-from-sql-to-nosql-225a
[3] I. Balbaert, The Way to Go: A Thorough Introduction to the Go Programming Language. 2012. [Online]. Available: https://openlibrary.org/books/OL25538095M/The_way_to_go

Research Gap

Cognitive Complexity

The MongoDB Go driver requires low-level coding, leading to repetitive and error-prone code for basic tasks like pagination and CRUD operations, which increases cognitive complexity.

Query Building

The absence of efficient mechanisms for building complex queries and managing cross-database transactions hinders performance and consistency across multiple databases.

Entity Validation

Existing Go ODMs do not offer built-in schema validation, forcing developers to write custom validation logic or rely on external libraries that do not integrate well with MongoDB.

Aggregation and Transactions Across Clusters

Existing solutions fail to handle data aggregation and transactions across multiple databases or clusters, which is crucial for distributed systems.

Plugin Architecture

Current ODMs lack a flexible plugin system to efficiently extend functionality, making it harder to customize or dynamically load features

Research Problem

Golang rise in popularity is hindered by the lack of a stable ODM (Object Document Mapper) to efficiently interact with NoSQL databases like MongoDB. Existing options like Mgo and MGM are outdated or insufficient for modern applications, forcing developers to rely on MongoDB official Go driver, which requires repetitive and error-prone low-level coding for basic tasks like CRUD operations and pagination. This increases cognitive complexity and reduces productivity. Additionally, Go lacks advanced ODM features common in other languages, such as schema validation, instance methods, and transaction management, which leads to fragmented code and a higher likelihood of bugs. The absence of a plugin architecture further limits customization and flexibility, while the lack of support for distributed transactions and cross-database aggregation impedes the development of scalable applications.The proposed Elemental system aims to address these gaps by providing a stable, feature-rich ODM with support for schema validation, plugin architecture, efficient query building, and distributed transaction management, ultimately enhancing the maintainability and scalability of applications built with Go and MongoDB.

Research Objectives

Efficient Query Builder for Go and Document Databases

The goal is to design a flexible and performant query builder in Go for document databases, enabling efficient data operations and intuitive query construction. This builder will support defining models and embedding query logic as instance methods, making it easy to build and execute complex queries within the model itself. Pagination features will allow efficient handling of large datasets, while soft deletes and cascading updates provide flexible data management options. Additionally, it will support population methods for joining related documents, improving query efficiency and readability.

Efficient Plugin Architecture and Data Pipeline for Plugins

This objective focuses on developing a modular plugin architecture that can efficiently handle and extend system functionality. By analyzing plugin-based architecture versus traditional monolithic designs, the aim is to determine the most effective approach, balancing modularity with performance. Safe hook points will be established to allow plugins to inject custom logic without compromising data consistency, while runtime context sharing will be carefully managed to maintain data integrity. An auditing mechanism will log plugin-driven modifications, ensuring traceability and accountability.

Mongo Driver Encapsulation and Pattern Implementation

Encapsulating the MongoDB driver within a robust, extensible pattern, this objective ensures that data handling is reliable and standardized across the application. The encapsulation will include schema validation to enforce data consistency and early-stage database validation to intercept errors before they reach the controller level. Additionally, migration support will streamline schema updates, and data seeding capabilities will facilitate easy setup and testing by preloading databases with required data. This approach creates a solid foundation for managing and evolving database structure and content.

Data Aggregation Across Clusters

This objective is to design a highly efficient system for aggregating data from multiple clusters, allowing concurrent access and transactional integrity across different databases. The solution will ensure data consistency and manage duplicates while writing to multiple sources, providing safe, concurrent read/write operations without risk of data loss. By analyzing existing concurrency algorithms and optimizing for our specific needs, the system will handle high-traffic operations smoothly, enabling seamless multi-cluster data aggregation and management.

Methodology

Task break down of Elemental is as followes,

1. Requirement Analysis: Identifies critical gaps in existing Go-based ODMs, focusing on performance, scalability, and developer experience.
2. System Design and Architecture: Organized into four layers—Core Layer, Query Builder Layer, Plugin Layer, and Validation and Concurrency Layer.
3. Core Component Development: Iterative development of key components such as the query builder, plugin system, validation system, and concurrency management.
4. Testing and Implementation: Involves benchmarking, integration testing, and user feedback to validate the system's performance and ensure it is scalable and ready for release.

Elemental’s development focused on creating a high-performance, modular Object Document Mapper for Go that simplifies MongoDB operations. Designed with a layered architecture, it combines an intuitive query builder, a flexible plugin system, and robust data validation to streamline database interactions. This structure enables extensibility, allowing easy customization without altering core functions, while the modular design enhances performance, maintainability, and scalability. Rigorous testing and benchmarking validated Elemental’s ability to handle high concurrency and complex queries, offering a reliable and developer-friendly solution for modern Go applications. The figure illustrates the system architecture and clear idea of separation and interaction of every functional layer.