DBChart and large tables

[Leroy Casterline]

BCB6 patch level 4, TeeChart 7.02 Pro.

I have a table whose records consist of 1 int and 5 doubles. The table
contains 1,000,000 records. This is a test application, real tables
could contain 20,000,000 records or more.

At runtime, I connect each of 5 series to the table, each with its
X-axis connected to the int and its Y-axis connected to one of the
doubles. Each series is in Dataset mode, with Dataset connected to my
Table.

1) When I set my table active at designtime, I get an out-of-memory
error. It seems that the DBChart is trying to load every point for each
series and is running out of memory.

2) At runtime, nothing displays in on my charts.

How can I set things up so that the DBChart buffers a small number of
points in memory and loads data as necessary?

If I can't do this, how can I make this work?

[Leroy Casterline]

Marjan, can you comment on this?

[Leroy Casterline]

Would someone please answer this?

[Marjan]

Hi, Leroy.

20,000,000 records or more

This is not problematic as far as number of points goes. But it might take a while to read all those records, populate series and finally draw chart.

When I set my table active at designtime, I get an out-of-memory

Strange, perhaps the problem is dataset changes before all series are populated and you end up with cyclic calls to CheckDataSource method. Try setting the TDBChart.AutoRefresh property to false. Another thing you might want to try is manually populate series with data (use "regular" TChart). Basically, do the same thing TDBChart does automatically. Here is an example:

Code: Select all

Query1.First;
Series1.Clear;
While not(Query1.EOF) do
begin
 Series1.AddXY(Query1.FieldByName('XValues').asInteger,
                       Query1.FieldByName('YValues').asDouble);
 Query1.Next;
end;

vthe DBChart buffers a small number of points in memory and loads data as necessary?

Hmm.. Perhaps it's better to use another query to limit/filter number of points and then connect it to series datasource. Then you can repopulate query (and refresh chart) as you move in the (big) original dataset.

[Leroy Casterline]

Hi, Marjan. Sorry to be slow in replying. I've been given a higher priority in a different part of the project for a few days.

Hi, Leroy.

Strange, perhaps the problem is dataset changes before all series are populated and you end up with cyclic calls to CheckDataSource method.

The dataset is completely populated and is static once populated, so its contents are not changing. Is this what you were refering to?

Try setting the TDBChart.AutoRefresh property to false.

I'll give this a try.

Another thing you might want to try is manually populate series with data (use "regular" TChart). Basically, do the same thing TDBChart does automatically. Here is an example:

Code: Select all

Query1.First;
Series1.Clear;
While not(Query1.EOF) do
begin
 Series1.AddXY(Query1.FieldByName('XValues').asInteger,
                       Query1.FieldByName('YValues').asDouble);
 Query1.Next;
end;

I'm already doing this. I was hoping to reduce complexity by having the chart manage the data.

Hmm.. Perhaps it's better to use another query to limit/filter number of points and then connect it to series datasource. Then you can repopulate query (and refresh chart) as you move in the (big) original dataset.

This sounds promising. I'm not much of a database programmer. Can you give me an example or tell me what I'd need to do to use this approach? I guess I'd have to use something other than a TTable?

[Marjan]

Hi, Leroy.

d, so its contents are not changing.

Exactly. Just ignore it if this is not the case.

me what I'd need to do to use this approach? I guess I'd have to use something other than a TTable?

Yes, I'd use a TQuery to "filter" which points I want on the TDBChart. Once the query is populated with data, you can connect it to TDBChart series in the same way as with TTable (or any other TDataset descendant).

[Leroy Casterline]

Hi, Leroy.
Yes, I'd use a TQuery to "filter" which points I want on the TDBChart. Once the query is populated with data, you can connect it to TDBChart series in the same way as with TTable (or any other TDataset descendant).

Thanks, Marjan. If I use this approach, can I then just leave the TQuery connected to the chart and change the query to display different data points? If I do it this way, will the data transition within the chart be visually smooth (without jumping or flashing)?

[Marjan]

Hi, Leroy.

can I then just leave the TQuery connected to the chart and change the query to display different data points

Yes. When query "generates" new data TDBChart will be automatically updated. Or you can manually force the repaint by calling series CheckDataSource method.

the chart be visually smooth (without jumping or flashing)?

It depends how many points you'll be including in query. Loading a TDBChart with 1M points will take some time <g>. On the other hand, if you'll be showing only small portion of points (1k), then the load operation should be quite fast.

[Leroy Casterline]

Hi, Leroy.

can I then just leave the TQuery connected to the chart and change the query to display different data points

Yes. When query "generates" new data TDBChart will be automatically updated. Or you can manually force the repaint by calling series CheckDataSource method.

Very good. Thanks for pointing out the CheckDataSource method. If I want to update manually, how do I tell the Chart not to automatically display the query result?

the chart be visually smooth (without jumping or flashing)?

It depends how many points you'll be including in query. Loading a TDBChart with 1M points will take some time <g>. On the other hand, if you'll be showing only small portion of points (1k), then the load operation should be quite fast.

I was thinking 2-3K points, so this sounds like it will work well. I started a test project last night.

Thanks for your help.

-Leroy