DATAFLOW: Weird graph in Estimated Cost feature from Dataflow

Hello people!

I love the Estimated cost feature since its a quick way to, not only get an estimated of the cost, but to compare to past version of the same job (a historic view would be nice!).

My question is:

Mostly, the graph looks like this, which is fine:

But this other graph looks weird, don’t you think?

Context:
Both graphs are from the same job but different runs.

First one Second one
Elapsed time
Succeeded

Running
Elapsed time
3 hr 59 min

3 hr 37 min
machine type n1-standard8 n1-standard8
job type batch batch
Job region us-central1 us-central1
Dataflow Prime Disabled Disabled
Runner v2 Enabled Enabled
Dataflow Shuffle Enabled Enabled

Current vCPUs

8
8
Total vCPU time

63.347 vCPU hr


27.761 vCPU hr
Total memory time
237.551 GB hr

104.188 GB hr
Current HDD PD
25 GB


25 GB

Total HDD PD time
197.959 GB hr

87.032 GB hr
Total Shuffle data processed
11.62 GB

4.88 GB
Billable Shuffle data processed 2.9 GB 1.22 GB


Best regards
David Regalado
Web | Linkedin | Twitter

Good day @davidregalado25 ,

Welcome back to Google Cloud Community!

Unfortunately, you can’t change this in the dataflow UI, its current visualization is set to Stacked Area Chart and you can only edit the time range for its charts and logs by clicking the dropdown (currently set to MAXTIME). However, you can edit the chart in the metrics explorer and save it to a dashboard by clicking the options button (three dots) and clicking (view in Metrics Explorer). You should be able to see different widget types (E.g. Line chart, stacked bar chart, etc.) that you can choose, and you can try editing the MQL query to your need, in this case, estimated cost per run.

You can use this link for more information: https://cloud.google.com/monitoring/mql/query-editor
https://cloud.google.com/monitoring/mql/query-language

Hope this helps!

1 Like

Hi!

Please, take a closer look, there are spaces with no data in that chart.


Best regards
David Regalado
Web | Linkedin | Twitter