<!--
  ~ Licensed to the Apache Software Foundation (ASF) under one
  ~ or more contributor license agreements.  See the NOTICE file
  ~ distributed with this work for additional information
  ~ regarding copyright ownership.  The ASF licenses this file
  ~ to you under the Apache License, Version 2.0 (the
  ~ "License"); you may not use this file except in compliance
  ~ with the License.  You may obtain a copy of the License at
  ~
  ~   http://www.apache.org/licenses/LICENSE-2.0
  ~
  ~ Unless required by applicable law or agreed to in writing,
  ~ software distributed under the License is distributed on an
  ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
  ~ KIND, either express or implied.  See the License for the
  ~ specific language governing permissions and limitations
  ~ under the License.
-->

# [ResLens](https://blog.resilientdb.com/2024/12/07/MemLens.html)

ResLens is a comprehensive CPU and memory profiler for ResilientDB which aims to provide granular performance metrics by integrating profiling tools like eBPF, node exporter, process exporter, LevelDB internal statistics and custom prometheus hooks. We aim to provide a platform which provides database specific metrics that provides detailed information on what happens under the hood and help optimize system performance. This is an ambitious idea which we are improving on constantly but are proud to share a packaged version of ResLens currently.

---

## Table of Contents
1. [Overview](#overview)
2. [Features](#features)
3. [Usage](#usage)
4. [Installation](#installation)
5. [Contributing](#contributing)
6. [License](#license)
7. [Acknowledgments](#acknowledgments)

---

## Overview

ResLens provides CPU, memory and build statistics aiming to provide detailed information on what happens under the hood and help optimize system performance.

### Key Highlights:
- Integrated terminal like playground to Get and Set values to/from `ResilientDB`.
- Real-time flame graph generation for memory and CPU profiling and detailed call stack analysis for processes.
- Custom CPP hooks integrated to provide metrics not exposed by LevelDB like cache hit ratio, storage engine metrics like number of SST tables, number of files at each level etc.
- Web-based frontend for interactive performance insights.
- Aggregating different profiling tools and packaging into a single dashboard.
- Lightweight and optimized for Linux-based systems.

---

## Features

- **Granular Profiling**: Analyze memory and CPU usage down to individual function calls.
- **Flame Graph Visualization**: Generate real-time flame graphs to identify bottlenecks.
- **Call Stack Recording**: Track and analyze call stacks during runtime.
- **User-Friendly Frontend**: Visualize results via a web-based interface. 
- **Dependency Analyzer**: Check dependency graphs generated by bazel and visualize them in a easy to view way. 
- **Output Options**: Export results in JSON, PNG, and other formats and analyze them using external tools.

---
# Usage
  ![Description](/docs/screenshots/flamegraph_get_method.png)
  **Description**:  Flamegraph showing the call stack when a Get method is called on ResilientDB's key value store. This graph shows the function caller which is invoked by the ResilientDB-GraphQL service and is serviced at the end by the storage engine which is LevelDB. The storage engine calls the in memory LRU cache to check for cache hits and when not found calls its internal functions to retrieve the key value pair.

  Todo: Add more examples

---

## Installation

### Prerequisites
- **Operating System**: Linux (e.g., Ubuntu 20.04+).
- **Announcement**: We know that these are a lot of tools to install and setup. Our team is working on automating the process of installation and setup.
- **Service Dependencies**:
  - Install our fork ResilientDB from here https://github.com/harish876/incubator-resilientDB
  - Install ResilientDB's Graphql service to expose HTTP endpoints to use the Get and Set values to the Key value store from here https://github.com/apache/incubator-resilientdb-graphql.
  - Install MemLen's middleware service to aggregate data from our profiling services and expose HTTP endpoints to allow the ResLens Dashboard to access this data. https://github.com/harish876/MemLens-middleware

- **System Dependencies**: These services are essential for hooking into various aspects of ResilientDB and monitoring its performance.

  - **Prometheus**: Expose metrics collected by services like `node_exporter` and `process-exporter` to the frontend.  
  - **NodeExporter**: Expose system-level metrics.  
  - **ProcessExporter**: Expose process-level metrics.  
  - **Pyroscope (v0.37.0)**: Provide fine-grained profiling metrics at a process level using eBPF.
  
### Steps to Run
 - Offline Mode - Learn, Explore and familiarize with the dashboard without setting up the profiling environment.
     1. Clone this repository. This is the frontend Dashboard to access the profiling data collected. Running the dashboard without an accompanying backend will display data profiled and collected by the team during the time of demo.
      ```bash
      git clone https://github.com/Bismanpal-Singh/MemLens

 - Online/Live Mode - Gauge metrics, optimise performance by looking at live metrics from the profiling environment.

    1. Clone this repository. This is the frontend Dashboard to access the profiling data collected.
      ```bash
      git clone https://github.com/Bismanpal-Singh/MemLens
      ```
    2. Clone the ResilientDB repository. This is the service which will be profiled by ResLens. Follow the installation instructions to install ResilientDB and set it up.
      ```bash
      git clone https://github.com/harish876/incubator-resilientDB
      ```
    3. Clone the ResilientDB Graphql repository. This is the service which will be used to Get and Set data from ResilientDB's key value store. Follow the installation instructions to install and setup this service.
      ```bash
      git clone https://github.com/apache/incubator-resilientdb-graphql.
      ```
    4. Clone the ResLens middleware service. This is the service which will be used to aggregate data from different profiling services.
      ```bash
      git clone https://github.com/harish876/MemLens-middleware
      ```
    5. Follow this step by step installation manual to install, setup, resolve errors, and configure the profiling environment.
      TODO ADD LINK

---
## Contribution
 1. Harish Gokul (http://github.com/harish876):
    Lead Developer for ResLens responsible for 
      - Ideation, Execution and Architecture Design for the project.
      - Development of CPP Hooks, integration of prometheus hooks, and third party profiling libraries.
      - Development of profiling environment
      - Development of ResLens middleware service 
      - Contributed heavily to the frontend dashboard design.
      - Documentation and blog author.

 1. Bismanpal Anand (http://github.com/bismanpal-singh):
    Designer & Developer for ResLens responsible for 
      - Structured and Designed the ResLens website.
      - Experimented with various approaches to enhance usability while maintaining simplicity and elegance.
      - Contributed to the design of the frontend dashboard, ensuring intuitive visualization of metrics and data.
      - Contributed in implementing the ResLens middleware service to bridge data collection and presentation.
      - Contributed heavily to the frontend dashboard design.
      - Gathered memory metrics and profiling data by working with APIs & tools and integrated them into the dashboard

----

## License
  TODO
---

## Acknowledgements

 - We would like to acknowledge the good folks of [ResView](https://github.com/ResilientApp/ResView) for the inspiration and a reference point to start our project.
