DataTable SelectionFeature - performance issue with many selected rows

[kguelzau]

Hi,

I am using a huge DataTable (20k rows with paging) with SelectionMode=multiple.
When I have selected many rows (> 1000) each paging request is getting really slow.

JProfiles shows that most of the time is spend in DataTable.getRowData() triggered by SelectionFeature.decodeMultipleSelection()
https://github.com/primefaces/primeface ... .java#L105
...leading to massive EL getValue calls.

My Question: Is it really necessary to decode on every paging request?
Shouldn't this be limited to requests containing a parameter javax.faces.behavior.event = rowSelectCheckbox || rowUnselectCheckbox || toggleSelect ?

Best Regards,

Kai Gülzau

[tandraschko]

AFAIR its always required when _selection request param is available as selection can also be applied via commandButton, which process the datatable, if no "selection p:ajax event" is required on the datatable.

[kguelzau]

Mhh OK. We do all our modifications via ajax, so this was out of sight

Another question regarding this issue.
Is it possible to speed things up in decodeMultipleSelection() by reusing RowData of the currently selected rows?
This way we only need to get the RowData for added rows and drop the data for removed rows.

Does it make sense? Is it possible?

[Babas007]

There is an option called "clientCache" for datatable, the description is not clear but you can give it a try

[kguelzau]

I think clientCache is just getting the next paging page in advance.
This will burn even more CPU on server side.

I just saw that getting the RowData for a specific rowKey is an O(n) operation instead of O(1) for the worst case:
https://github.com/primefaces/primeface ... .java#L817

So with 1000 selected rows in a 20k table we end up in 1000 * (20000/2) ~ 10 Million getRowKey EL evaluations!

There must be a either a smarter way to get the rowData for a rowKey or
decodeMultipleSelection() should only act on changed items.

So after all it seems to be a solution to use a DataModel which implements SelectableDataModel or extends LazyDataModel.
This way the rowData is provided directly by the model.

Using a dataTable >1000 selectable items without this kind of model is really imperformant.

[tandraschko]

You don't use LazyDataModel with such amount of rows?
Please try LazyDataModel and come back if you still have performance problems.

[kguelzau]

You don't use LazyDataModel with such amount of rows?

Why should I do this when all my data is already in memory and I use a classic paging?
It's not very clear in the documentation.
For me LazyDataModel was the solution when loading data is expensive...

Lazy Loading is an approach to deal with huge datasets efficiently, regular ajax based pagination
works by rendering only a particular page but still requires all data to be loaded into memory. Lazy
loading datatable renders a particular page similarly but also only loads the page data into memory
not the whole dataset. In order to implement this, you'd need to bind a
org.primefaces.model.LazyDataModel as the value and implement load method and enable lazy
option. Also it is required to implement getRowData and getRowKey if you have selection enabled.

As a suggestion. Extend the documentation of the "value" Attribute of dataTable
and give a hint to use SelectableDataModel / LazyDataModel for Tables using the Multiselect Feature.

Performance is good OK, but there is still the bad case of O(n) while e.g. paging when I have selected almost all rows.

[tandraschko]

DataTable is just not optimized for this case and mostly a bad pattern.
Users likely use ViewScoped and sometimes enabled serialization -> leads to performance problems with very very big state.
Of course, everything can be optimized but we just recommend: Big data -> LazyDataModel

If there is something to improve our performance for selection and lazyDataModel, free to create a issue + analysis + maybe a patch.

You can also create a ticket for improving the documentation - i don't have access to it.