Looker cloud 5000 limit

Looker Q&A Looker Studio
, , , , , , , , , , ,
1

Title is saying it all, why does looker have a limit of 5k rows?

If I use an explore with let’s say 500K records, then I as an dashboard developer want to choose if there is an limit.

I can understand that looker is Quary based so 5K records maximum gives faster load times. but why is this enforced so heavily? like i said above. If I want to show 500K in a detail’s table where users can scroll through then I think it should be possible. other products like Qlik and PowerBI have this feature natively.

2

I’m going to assume you’re talking about Looker and not Looker Studio! In Looker Studio, tables don’t have any such row limit.

In Looker, this row limit has historically been more of a browser limitation than a query limitation. We’ve found that the more cells that there are on the screen, the slower the browser goes. In fact, if you have 5000 rows and simply add enough columns, you can crash your browser pretty easily. I guess the Looker interface isn’t really optimized for browsing through large amounts of data.

The solution I’ve always recommended is to download the data and browse the downloaded file.

3

@sam8 Thanks for the response,

but this counter-acts the purpose of BI-tooling. the idea of BI-tooling is that user in general do not use (or a lot less) an excel spreadsheet (or google-sheets) because of the fact that it is to easily manipulated. also the 1-truth principle, if exporting is the solution here is, is at stake here.

and if the looker-program is browser optimized, that it might be time to update the program.

other BI-tools like Qlik, I can have a lot of data in a table without having to worry about crashing my browser.

@sam8 I want to know you point of view on this. let’s say i’m a user and I have a dashboard in Looker. I scroll the dashboard and see a detailed (table with 5K rows maximum) then I can assume 2 things, ooh 5k rows might be all the data or why am I limited to 5K rows I want to explore the data more, interact with it. select (filter) a couple of anomalies I see and check other visuals in the same dashboard because of those anomalies.

4

You make some excellent points. I’ll engage to the best of my ability, but I want to caveat that I am NOT a product manager, and my takes on the product vision could be skewed by my experience in Looker Support.

Overall - I agree with you. I used to think that browsing through more than 5000 rows was a niche and overly laborious use case, but I’ve been proven to be quite mistaken on that front. The fact that other BI tools, even Looker Studio, can handle this without any issues tells us that it’s a valid use case and there’s not really a good “reason” for Looker not to handle it. It’s been this way since the product’s inception, and my guess is that there just hasn’t been quite enough demand for this feature to get prioritized.

On the other hand, I think you’re selling the export solution a bit short. Just because you can manipulate exported data doesn’t undermine the entire governance principle of a BI tool. The export still came from governed data. Would you also argue that table calculations undermine a BI tool? In theory, I could manipulate the data any way I wanted using table calculations and then hide all of the original dimensions and measures. My point is that there is still a lot of value in having everyone start from the same data model.

That said - your points are super valid and well taken. I’d definitely recommend creating or upvoting a feature request in the product about this! The more we hear about it, the more likely we are to take action on it.

5

@sam8
Thank you again for the response, this shine some light on it all.

But I wonder if you or other people on the Looker team have ever been working in the field of BI. Please don’t see this as an insult.

Why Am I saying this:

In my experiance working as both a BI-consultant and a BI-developer I have talked to different layers in organisations (operational end users to (upper-)managers) all need differtent types of data to be shown in there dashboards at different levels (aggerarations). But I have notice one trend for all of them. They all like to get an overview in the product itself. They don’t want to export right away. but usually scroll down (in a table) to spot anomalies, check other visualisations (charts, maps). If they have spotted some anomalies they will filter the data to get a better view of why that anomaly is there. after all of that, in the end the user want to export there data.

Export button

I think it is great that there is an export button (that should be a minimum for every BI-product)

What often happens is that people export out of the BI-tool this is indeed governed data. but then the user starts to add there own data, manipulating stuff (vlook-up, etc.) then that file will usually be updated with or without governed data and in the end the data is shared with other people in the company. now you got 2 sources 1 goverened and 1 semi-goverened

That is true, but usually, in my experance, people don’t always know what they are doing. As an example: we start from the same datamodel. 1 measue and 1 dimension.
DIM: orderID

Measure: sum distinct revenue.

if for some reason, and that happens more the you think the souce database (for looker or as a general source system) is not cleaned (with deleting duplicates) then in looker we got for every orderID the right revenue. but if a user have acces (read rights) to the source system (like an ordersystem) then he might find that the revenue for orderID Y has to be 3 times as high (because that ordersystem has duplicated the order 2 times) therefore now you got 2 truths 1 from Looker (the real truth) and 1 from an excel that has been changed by the end-user.

also note that not everybody know’s what a sum/count/min/max does, then seeing 2 different numbers triggers those people into thinking there is something wrong. Ususally they think that there calculation is the right one.

I wonder if this will actually do something. to increase the 5K limit have been asked before, but until now it never has been changed. :thinking:

just out of curiosity, is this because the development-team for looker is just to small to handle bigger tickets? Because I would assume increasing the 5K limit will be a very big change.

Once agian, I hope you and the looker team don’t take this as a personal assault. But these types of limit’s can be dealbreakers for many companies.

6

@Mart01

Completely agree on the critical use case of row limits in BI for a variety of use cases, this has been a limitation Looker customers have been pushing the product on for quite some time.

We’re expecting this will be a very large improvement targeting the end of Q3 for customer preview. We have a committed roadmap project to increase this limit exponentially as well as having increases particularly for scatter plot and geo map viz types.

This should be a pretty huge improvement for a widely requested feature, and preview for this is targeted to end of Q3.

3 Likes
7

@SZinsmeister Thanks for that proposed update. is there anywhere i can see the roadmap?

8

@Mart01 roadmap refresh is coming up following our NEXT 2025 user conference which is underway this week. I recommend reaching out to your account manager and we can setup a session with the right product specialists to walk you through the roadmap, get your feedback, and answer any questions you might have. We also have a roadmap digital event coming up in May that will be posted in a few weeks.

1 Like