Hi Guys, what is the best practice to handle large google sheets tables in excess of 100 000 rows? I’m wanting to pull in a sheet of transactional sales data that is extremely long, but having all of the data is key to some of the machine learning practices I am wanting to try and achieve…
I have been playing around with some ideas in this area.
First, for basic management of the transactions, you likely do not need all of the history. I would recommend using Security Filters to trim down the rows for more recent history only.
Andre_van_Aarde:
but having all of the data is key to some of the machine learning practices I am wanting to try and achieve…
I’m curious what kind machine learning you are applying that you would need 100000 rows?
One of my concerns is charting against history, without pulling in all the history rows. The charts deal mainly with summarized information for daily, weekly, monthly, quarterly, etc. My thought here is capture the summarized details in their own table that remains mostly static. A challenge is dealing with situations such as corrections to the history.
I am then toying with the idea of placing history in a separate sheet and building separate apps, one each for history and reporting/charts.
Scripting services would likely come into play heavily to manage the behind the scenes movement of the data between sheets.
I have a similar issue where I have less than 150K rows and is not expected to grow.
Kingly check my feature request and support if it makes sense to you.
Combining Multiple Tables based on Google Sheet as one list/table (Append)
Use Case :> A table based on a Google Sheet with less than 150,000 rows.> > Limitations encountered :> Appsheet can only connect to a Google Sheet with 100,000 rows or less> > Available Workaround :> Split the Google Sheet into two worksheets, making sure each worksheet doesn’t have more than 100,000 rows.> > - Import the two worksheets as separate tables> - Use each worksheet as data partitions> > Problems with the available workarounds> Users will need full access to the list, and switching between tables or partitions by using USERSETTINGS() or table expressions puts some limitations on some use cases where the tables are used as source for adding items to a child table using ref_rows (sub-form).