Feat: Add Custom OpenTelemetry Exporter in for Service Metrics #2272
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Note This is a re-submission of #2270 with some additional fixes done, please close #2270 and I'll continue to address issues from CI tests & review comments here instead (as @ravjotbrar is away on vacation, and I don't have access to his fork). |
6 tasks
surbhigarg92
reviewed
Apr 4, 2025
surbhigarg92
reviewed
Apr 4, 2025
surbhigarg92
reviewed
Apr 8, 2025
src/metrics/transform.ts
Outdated
| dataPointType === DataPointType.EXPONENTIAL_HISTOGRAM | ||
| ) { | ||
| return ValueType.DISTRIBUTION; | ||
| } else if (name.includes('count')) { |
There was a problem hiding this comment.
Except this everything else is generic and auto detected. If possible , we could make this generic, if this would add a lot to code maintainability please skip.
There was a problem hiding this comment.
At the moment, I'm unable to think of solutions that would make this generic. If I get an idea, I will definitely make the change.
There was a problem hiding this comment.
I updated this to look at the data point type (SUM = INT64, GAUGE = Double, histogram types remain the same) instead of the name.
a2c33ff to
bdb207d
Compare
surbhigarg92
previously approved these changes
Apr 11, 2025
sakthivelmanii
previously approved these changes
Apr 11, 2025
This comment was marked as outdated.
This comment was marked as outdated.
|
Merge-on-green attempted to merge your PR for 6 hours, but it was not mergeable because either one of your required status checks failed, one of your required reviews was not approved, or there is a do not merge label. Learn more about your required status checks here: https://help.github.com/en/github/administering-a-repository/enabling-required-status-checks. You can remove and reapply the label to re-run the bot. |
eaac697 to
9ef1e71
Compare
77fb37f to
15dcaf9
Compare
15dcaf9 to
510408d
Compare
surbhigarg92
previously approved these changes
May 22, 2025
| assert(sendTimeSeriesStub.notCalled); | ||
| }); | ||
|
|
||
| it('should handle failed send during time series export with callback', async () => { |
There was a problem hiding this comment.
nit: can we also add a test case for the max_batch_export_size that createTimeSeries is called multiple times as per the batch size
There was a problem hiding this comment.
Added test for batches below.
133e0e0 to
9c3771d
Compare
surbhigarg92
approved these changes
May 23, 2025
6 tasks
gcf-merge-on-green bot
pushed a commit
that referenced
this pull request
Jul 29, 2025
🤖 I have created a release *beep* *boop* --- ## [8.1.0](https://togithub.com/googleapis/nodejs-spanner/compare/v8.0.0...v8.1.0) (2025-07-28) ### Features * Add Custom OpenTelemetry Exporter in for Service Metrics ([#2272](https://togithub.com/googleapis/nodejs-spanner/issues/2272)) ([610d1b9](https://togithub.com/googleapis/nodejs-spanner/commit/610d1b989ba186c0758791343deaa7f683c4bd26)) * Add methods from gax to cache proto root and process custom error details ([#2330](https://togithub.com/googleapis/nodejs-spanner/issues/2330)) ([1b3931a](https://togithub.com/googleapis/nodejs-spanner/commit/1b3931a799bdd052adc91703e59e1d0c83270065)) * Add metrics tracers ([#2319](https://togithub.com/googleapis/nodejs-spanner/issues/2319)) ([192bf2b](https://togithub.com/googleapis/nodejs-spanner/commit/192bf2bb603bca4ac481fcfd1f04974173adc6a1)) * Add support for AFE latency metrics ([#2348](https://togithub.com/googleapis/nodejs-spanner/issues/2348)) ([0666f05](https://togithub.com/googleapis/nodejs-spanner/commit/0666f05d589e2f229b44dffae8e9649220bccf8b)) * Add throughput_mode to UpdateDatabaseDdlRequest to be used by Spanner Migration Tool. See https://togithub.com/GoogleCloudPlatform/spanner-migration-tool ([#2304](https://togithub.com/googleapis/nodejs-spanner/issues/2304)) ([a29af56](https://togithub.com/googleapis/nodejs-spanner/commit/a29af56ae3c31f07115cb938bcf3f0f77241b725)) * Operation, Attempt, and GFE metrics ([#2328](https://togithub.com/googleapis/nodejs-spanner/issues/2328)) ([646e6ea](https://togithub.com/googleapis/nodejs-spanner/commit/646e6ea6f1dc5fa1937e512ae9e81ae4d2637ed0)) * Proto changes for an internal api ([#2356](https://togithub.com/googleapis/nodejs-spanner/issues/2356)) ([380e770](https://togithub.com/googleapis/nodejs-spanner/commit/380e7705a23a692168db386ba5426c91bf1587b6)) * **spanner:** A new field `snapshot_timestamp` is added to message `.google.spanner.v1.CommitResponse` ([#2350](https://togithub.com/googleapis/nodejs-spanner/issues/2350)) ([0875cd8](https://togithub.com/googleapis/nodejs-spanner/commit/0875cd82e99fa6c95ab38807e09c5921303775f8)) * **spanner:** Add new change_stream.proto ([#2315](https://togithub.com/googleapis/nodejs-spanner/issues/2315)) ([57d67be](https://togithub.com/googleapis/nodejs-spanner/commit/57d67be2e3b6d6ac2a8a903acf8613b27a049c3b)) * **spanner:** Add tpc support ([#2333](https://togithub.com/googleapis/nodejs-spanner/issues/2333)) ([a381cab](https://togithub.com/googleapis/nodejs-spanner/commit/a381cab92c31373a6a10edca0f8a8bdfc4415e4b)) * Track precommit token in r/w apis(multiplexed session) ([#2312](https://togithub.com/googleapis/nodejs-spanner/issues/2312)) ([3676bfa](https://togithub.com/googleapis/nodejs-spanner/commit/3676bfa60725c43f85a04ead87943be92e4a99f0)) ### Bug Fixes * Docs-test ([#2297](https://togithub.com/googleapis/nodejs-spanner/issues/2297)) ([61c571c](https://togithub.com/googleapis/nodejs-spanner/commit/61c571c729c2a065df6ff166db784a6e6eaef74d)) * Ensure context propagation works in Node.js 22 with async/await ([#2326](https://togithub.com/googleapis/nodejs-spanner/issues/2326)) ([e8cdbed](https://togithub.com/googleapis/nodejs-spanner/commit/e8cdbedd55f049b8c7766e97388ed045fedd1b4e)) * Pass the Span correctly ([#2332](https://togithub.com/googleapis/nodejs-spanner/issues/2332)) ([edaee77](https://togithub.com/googleapis/nodejs-spanner/commit/edaee7791b2d814f749ed35119dd705924984a78)) * System test against emulator ([#2339](https://togithub.com/googleapis/nodejs-spanner/issues/2339)) ([2a6af4c](https://togithub.com/googleapis/nodejs-spanner/commit/2a6af4c36484f44929a2fac80d8f225dad5d702c)) * Unhandled exceptions from gax ([#2338](https://togithub.com/googleapis/nodejs-spanner/issues/2338)) ([6428bcd](https://togithub.com/googleapis/nodejs-spanner/commit/6428bcd2980852c1bdbc4c3d0ab210a139e5f193)) ### Performance Improvements * Skip gRPC trailers for StreamingRead & ExecuteStreamingSql ([#2313](https://togithub.com/googleapis/nodejs-spanner/issues/2313)) ([8bd0781](https://togithub.com/googleapis/nodejs-spanner/commit/8bd0781e8b434a421f0e0f3395439a5a86c7847c)) --- This PR was generated with [Release Please](https://togithub.com/googleapis/release-please). See [documentation](https://togithub.com/googleapis/release-please#release-please).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This change adds an implementation for an OpenTelemetry exporter to send the following service metrics that are already present in the Python, Go and Java clients:
operation_latencies operation_count attempt_latencies attempt_count gfe_latency gfe_missing_header_count
Files added to a metrics folder in /src/metrics/ folder:
constants.ts - Constant values such as metric and label names to be used by both the exporter and eventual implementing code. spanner-metrics-exporter.ts - The definition for the exporter external-types.ts - Type definition for ExporterOptions, MetricKind, and ValueType transform.ts - Methods that help create a GCM timeseries list from scope metrics.
Based on regular client Google Cloud Monitoring exporter found here
Impact
Integrates Built-in Metrics support into Node.js spanner client library.
Testing
Unit tests were added in /test/metrics
Checklist
Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
Ensure the tests and linter pass
Code coverage does not decrease
Appropriate docs were updated
Appropriate comments were added, particularly in complex areas or places that require background
No new warnings or issues will be generated from this change