Skip to content

Commit e41d93a

Browse files
surbhigarg92surbhigarg92
committed
benchmarking results
1 parent 10f14a9 commit e41d93a

5 files changed

Lines changed: 183 additions & 66 deletions

File tree

‎.gitignore‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,5 @@ gorm/
33
.DS_Store
44
node_modules
55
build/
6-
package-lock.json
6+
package-lock.json
7+
googleapis/

‎spannerlib/.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,3 +10,5 @@ vendor/bundle
1010
*.swp
1111
ext/
1212
Gemfile.lock
13+
/googleapis/
14+
package-lock.json

‎spannerlib/wrappers/spannerlib-nodejs-poc/benchmark/README.md‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# Spanner Node.js Wrappers Benchmark
2+
3+
This directory contains a benchmark suite to compare the performance, throughput, and memory usage of three different Node.js wrappers for the Spanner Shared C-Library:
4+
1. **Koffi** (Fast FFI)
5+
2. **N-API** (Native C++ Addon)
6+
3. **IPC** (gRPC Server/Client)
7+
8+
## Prerequisites
9+
Ensure you have the following installed on the machine:
10+
* **Node.js** (v22 recommended)
11+
* **Go** (v1.22 recommended)
12+
* **Build Tools** (`gcc`, `g++`, `make`)
13+
14+
---
15+
16+
## Setup Instructions
17+
18+
### 1. Fetch Google API Protos (For IPC)
19+
The IPC wrapper depends on standard Google API protos. You can fetch them by running the build script in the `grpc-server` directory:
20+
21+
```bash
22+
cd spannerlib/grpc-server
23+
./build-protos.sh
24+
```
25+
*(Note: This script also attempts to generate Go code using `protoc`. If you do not have `protoc` installed, it may show an error on the generation step, but it will still have successfully cloned the `googleapis` folder that Node.js needs).*
26+
27+
### 2. Build the Go Shared Library
28+
Navigate to the shared library directory and build the `.so` file:
29+
```bash
30+
cd spannerlib/shared
31+
go build -o libspanner.so -buildmode=c-shared shared_lib.go
32+
```
33+
34+
### 3. Install Dependencies for Wrappers
35+
Navigate to each wrapper directory and install its specific dependencies:
36+
37+
**Koffi:**
38+
```bash
39+
cd spannerlib/wrappers/spannerlib-nodejs-poc/koffi
40+
npm install
41+
```
42+
43+
**N-API:**
44+
```bash
45+
cd spannerlib/wrappers/spannerlib-nodejs-poc/napi
46+
npm install # This will also automatically build the C++ addon
47+
```
48+
49+
**IPC:**
50+
```bash
51+
cd spannerlib/wrappers/spannerlib-nodejs-poc/ipc
52+
npm install
53+
```
54+
55+
### 4. Start the gRPC Server (Required for IPC)
56+
The IPC wrapper requires the Go gRPC server to be running.
57+
Navigate to `spannerlib/grpc-server` and run it in the background (or in a `tmux` session):
58+
```bash
59+
cd spannerlib/grpc-server
60+
nohup go run server.go localhost:50051 tcp > server.log 2>&1 &
61+
```
62+
63+
---
64+
65+
## Running the Benchmark
66+
67+
Once all setups are complete, navigate to this directory:
68+
69+
```bash
70+
cd spannerlib/wrappers/spannerlib-nodejs-poc/benchmark
71+
npm install
72+
node --expose-gc benchmark.js
73+
```
74+
75+
## Results
76+
The script will generate a `RESULTS.md` file in this directory containing:
77+
* Latency statistics from `mitata`.
78+
* Throughput (ops/sec) and Heap memory delta for 5 runs of each wrapper.
79+
* A 1-minute cooldown sleep is automatically applied between wrappers to ensure DB cooling and isolation.

‎spannerlib/wrappers/spannerlib-nodejs-poc/benchmark/RESULTS.md‎

Lines changed: 24 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -4,58 +4,52 @@
44
- **Database**: `projects/span-cloud-testing/instances/gargsurbhi-testing/databases/test-db-koffi-136`
55

66
## Mitata Latency Benchmark
7-
8-
```
9-
cpu: Apple M4 Pro
10-
runtime: node v22.17.1 (arm64-darwin)
11-
127
benchmark time (avg) (min … max) p75 p99 p999
138
--------------------------------------------------------------- -----------------------------
14-
Koffi Single Point Read 66'049 µs/iter (62'838 µs … 69'484 µs) 68'705 µs 69'484 µs 69'484 µs
15-
N-API Single Point Read 71'349 µs/iter (66'112 µs … 83'420 µs) 76'782 µs 83'420 µs 83'420 µs
16-
IPC Single Point Read 75'546 µs/iter (66'407 µs … 97'423 µs) 91'583 µs 97'423 µs 97'423 µs
17-
```
9+
Koffi Single Point Read 4'228 µs/iter (2'763 µs … 7'062 µs) 4'795 µs 6'234 µs 7'062 µs
10+
N-API Single Point Read 4'223 µs/iter (2'745 µs … 6'530 µs) 4'777 µs 5'694 µs 6'530 µs
11+
IPC Single Point Read 6'744 µs/iter (5'166 µs … 11'075 µs) 7'296 µs 11'075 µs 11'075 µs
1812

1913
## Summary of Manual Benchmark (Average of 5 Runs)
2014

2115
| Wrapper | Throughput | Avg Latency | Min Latency | Max Latency | Heap Diff (Avg) |
2216
| :--- | :--- | :--- | :--- | :--- | :--- |
23-
| **Koffi** | 13.83 ops/sec | 72.48 ms | 60.38 ms | 159.86 ms | 0.73 MB |
24-
| **N-API** | 13.97 ops/sec | 71.81 ms | 59.34 ms | 267.20 ms | 0.55 MB |
25-
| **IPC** | 13.24 ops/sec | 75.80 ms | 63.78 ms | 215.10 ms | 13.28 MB |
17+
| **Koffi** | 247.50 ops/sec | 4.04 ms | 2.19 ms | 120.20 ms | 0.48 MB |
18+
| **N-API** | 251.79 ops/sec | 3.97 ms | 2.08 ms | 114.84 ms | 0.29 MB |
19+
| **IPC** | 178.48 ops/sec | 5.60 ms | 3.55 ms | 62.48 ms | 1.11 MB |
2620

2721
## Detailed Runs
2822

2923
### Koffi
3024

3125
| Run | Start Time | End Time | Throughput | Avg Latency | Min Latency | Max Latency | Heap Diff |
3226
| :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- |
33-
| Run 1 | 2026-04-13T06:26:00.195Z | 2026-04-13T06:26:35.666Z | 14.10 ops/sec | 70.94 ms | 60.02 ms | 369.14 ms | 0.68 MB |
34-
| Run 2 | 2026-04-13T06:26:37.677Z | 2026-04-13T06:27:11.543Z | 14.76 ops/sec | 67.73 ms | 59.99 ms | 95.69 ms | 0.76 MB |
35-
| Run 3 | 2026-04-13T06:27:13.556Z | 2026-04-13T06:27:51.062Z | 13.33 ops/sec | 75.01 ms | 61.05 ms | 108.63 ms | 0.72 MB |
36-
| Run 4 | 2026-04-13T06:27:53.073Z | 2026-04-13T06:28:31.692Z | 12.95 ops/sec | 77.24 ms | 61.14 ms | 106.80 ms | 0.71 MB |
37-
| Run 5 | 2026-04-13T06:28:33.703Z | 2026-04-13T06:29:09.433Z | 13.99 ops/sec | 71.46 ms | 59.70 ms | 119.04 ms | 0.78 MB |
38-
| **Average** | - | - | 13.83 ops/sec | 72.48 ms | 60.38 ms | 159.86 ms | 0.73 MB |
27+
| Run 1 | 2026-04-13T16:09:56.696Z | 2026-04-13T16:13:17.847Z | 248.57 ops/sec | 4.02 ms | 2.10 ms | 211.72 ms | 0.63 MB |
28+
| Run 2 | 2026-04-13T16:13:19.850Z | 2026-04-13T16:16:39.698Z | 250.19 ops/sec | 4.00 ms | 2.21 ms | 50.32 ms | 0.60 MB |
29+
| Run 3 | 2026-04-13T16:16:41.700Z | 2026-04-13T16:20:04.955Z | 246.00 ops/sec | 4.06 ms | 2.22 ms | 46.13 ms | 0.50 MB |
30+
| Run 4 | 2026-04-13T16:20:06.957Z | 2026-04-13T16:23:30.452Z | 245.71 ops/sec | 4.07 ms | 2.23 ms | 83.57 ms | 0.59 MB |
31+
| Run 5 | 2026-04-13T16:23:32.455Z | 2026-04-13T16:26:54.852Z | 247.04 ops/sec | 4.05 ms | 2.18 ms | 209.29 ms | 0.05 MB |
32+
| **Average** | - | - | 247.50 ops/sec | 4.04 ms | 2.19 ms | 120.20 ms | 0.48 MB |
3933

4034
### N-API
4135

4236
| Run | Start Time | End Time | Throughput | Avg Latency | Min Latency | Max Latency | Heap Diff |
4337
| :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- |
44-
| Run 1 | 2026-04-13T06:30:09.445Z | 2026-04-13T06:30:45.556Z | 13.85 ops/sec | 72.22 ms | 59.20 ms | 633.28 ms | 0.56 MB |
45-
| Run 2 | 2026-04-13T06:30:47.561Z | 2026-04-13T06:31:20.317Z | 15.26 ops/sec | 65.51 ms | 59.25 ms | 131.44 ms | 0.53 MB |
46-
| Run 3 | 2026-04-13T06:31:22.329Z | 2026-04-13T06:31:57.787Z | 14.10 ops/sec | 70.92 ms | 58.95 ms | 129.25 ms | 0.56 MB |
47-
| Run 4 | 2026-04-13T06:31:59.797Z | 2026-04-13T06:32:36.084Z | 13.78 ops/sec | 72.57 ms | 59.58 ms | 192.24 ms | 0.55 MB |
48-
| Run 5 | 2026-04-13T06:32:38.096Z | 2026-04-13T06:33:17.018Z | 12.85 ops/sec | 77.84 ms | 59.74 ms | 249.79 ms | 0.55 MB |
49-
| **Average** | - | - | 13.97 ops/sec | 71.81 ms | 59.34 ms | 267.20 ms | 0.55 MB |
38+
| Run 1 | 2026-04-13T16:27:56.860Z | 2026-04-13T16:31:15.625Z | 251.55 ops/sec | 3.97 ms | 2.07 ms | 212.07 ms | -1.32 MB |
39+
| Run 2 | 2026-04-13T16:31:17.628Z | 2026-04-13T16:34:37.128Z | 250.63 ops/sec | 3.99 ms | 2.07 ms | 211.28 ms | 0.60 MB |
40+
| Run 3 | 2026-04-13T16:34:39.131Z | 2026-04-13T16:37:58.728Z | 250.50 ops/sec | 3.99 ms | 2.10 ms | 49.52 ms | -0.15 MB |
41+
| Run 4 | 2026-04-13T16:38:00.729Z | 2026-04-13T16:41:19.177Z | 251.96 ops/sec | 3.97 ms | 2.10 ms | 48.44 ms | -0.19 MB |
42+
| Run 5 | 2026-04-13T16:41:21.180Z | 2026-04-13T16:44:37.807Z | 254.29 ops/sec | 3.93 ms | 2.07 ms | 52.89 ms | 2.50 MB |
43+
| **Average** | - | - | 251.79 ops/sec | 3.97 ms | 2.08 ms | 114.84 ms | 0.29 MB |
5044

5145
### IPC
5246

5347
| Run | Start Time | End Time | Throughput | Avg Latency | Min Latency | Max Latency | Heap Diff |
5448
| :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- |
55-
| Run 1 | 2026-04-13T06:34:17.030Z | 2026-04-13T06:34:58.429Z | 12.08 ops/sec | 82.80 ms | 63.94 ms | 608.08 ms | 8.40 MB |
56-
| Run 2 | 2026-04-13T06:35:00.440Z | 2026-04-13T06:35:39.422Z | 12.83 ops/sec | 77.96 ms | 64.92 ms | 134.13 ms | 15.16 MB |
57-
| Run 3 | 2026-04-13T06:35:41.437Z | 2026-04-13T06:36:17.121Z | 14.01 ops/sec | 71.37 ms | 62.60 ms | 112.61 ms | 15.49 MB |
58-
| Run 4 | 2026-04-13T06:36:19.130Z | 2026-04-13T06:36:54.629Z | 14.09 ops/sec | 71.00 ms | 62.87 ms | 118.40 ms | 13.68 MB |
59-
| Run 5 | 2026-04-13T06:36:56.642Z | 2026-04-13T06:37:34.589Z | 13.18 ops/sec | 75.89 ms | 64.54 ms | 102.27 ms | 13.66 MB |
60-
| **Average** | - | - | 13.24 ops/sec | 75.80 ms | 63.78 ms | 215.10 ms | 13.28 MB |
49+
| Run 1 | 2026-04-13T16:45:39.821Z | 2026-04-13T16:50:15.184Z | 181.58 ops/sec | 5.51 ms | 3.54 ms | 100.94 ms | 7.74 MB |
50+
| Run 2 | 2026-04-13T16:50:17.187Z | 2026-04-13T16:54:54.569Z | 180.26 ops/sec | 5.55 ms | 3.56 ms | 53.70 ms | -0.07 MB |
51+
| Run 3 | 2026-04-13T16:54:56.572Z | 2026-04-13T16:59:37.603Z | 177.92 ops/sec | 5.62 ms | 3.57 ms | 52.15 ms | -0.16 MB |
52+
| Run 4 | 2026-04-13T16:59:39.606Z | 2026-04-13T17:04:22.795Z | 176.56 ops/sec | 5.66 ms | 3.57 ms | 55.32 ms | -5.46 MB |
53+
| Run 5 | 2026-04-13T17:04:24.798Z | 2026-04-13T17:09:08.731Z | 176.10 ops/sec | 5.68 ms | 3.50 ms | 50.27 ms | 3.48 MB |
54+
| **Average** | - | - | 178.48 ops/sec | 5.60 ms | 3.55 ms | 62.48 ms | 1.11 MB |
6155

0 commit comments

Comments
 (0)