Steema Central

Our web server app uses teeChart to create charts on the server and then display them on the browser via the load from URL. We have been having IIS hang occaisonally. This is a particular problem with IIS 6.0. When this happens and we collect an IISState dump of the w3wp process, the state dump is always the same. A bunch of threads are waiting on a particular thread which has teeChart6 invoking SendMessageA. It doesn't seem like this should be happening under a Web Server.

Under what context's would teeChart6 be invoking SendMessageA?

Is there some configuration setting to tell teeChart not to invoke user interface API's?

I've seen the FAQ which mentions about the trial version causing IIS to hang, but this does not apply to us as we have been running teeChart for years.

I forgot to add the teeChart version, 6.0.0.6.

Hi,

it's hard to know which the problem is without being able to reproduce it here ? Does it happens in a special situation ? Do you have any asp example withi which we can reproduce as is the problem here ?

I agree that's it's hard to know the problem. Unfortunately we have been unable to reproduce this in a small test case. I'm looking for some help in pursuing what we do know.

The problem was happening several times a week when one developer felt that he narrowed the problem down to the import.LoadFromStream call. We created a VB COM object with a single interface as follows:
Public Function loadChartFromStream( _
ByRef objChart As Variant, _
ByVal binData As Variant _
)

objChart.Import.LoadFromStream binData
End Function
We made this object a single threaded apartment threading model to serialize all access to the LoadFromStream method.
Then we changed every call to LoadFromStream to use the COM object method. This drastically reduced the number of hangs we were experiencing. We still experience a hang about once every 14 to 21 days. We have not always been consistent in capturing the IISState dump, but when we have the thread that all the other threads are waiting for shows teeChart as the last thing on the stack invoking the SendMessageA method in user32.dll. I thought maybe tracking down the code paths that cause the invocation of SendMesssageA might shed some light on the problem or a potential workaround. I realize this may not be a straigtforward request.

Hello,

We'll see if we can recreate this problem. Just to clarify:

The IIS based app puts a TeeChart on a page and that Chart loads a tee file via LoadFromURL in native form and LoadFromStream when wrapped for tests on a VB UserControl? When using LoadFromStream how are you sourcing the stream?

When using LoadFromURL are you running against a repository of existing tee files or are those files generated on the fly at call time?

Does the application hang whilst the load is running or have you not yet been able to identify the exact failure moment?

If you have a mockup example or code that could help us rebuild/reproduce the application here, plus any excerpt from the error log please put it on the news://www.steema.net/steema.public.attachments newsgroup and we'll see if we can reproduce the error.

What is your server platform (hardware/software)?

With thanks.
Regards,
Marc Meumann

Thanks for trying this out.

The primary flow for creating and display charts is:
Server side:
- teeChart obj is instantiated
- teeChart is initialized with default chart via import.LoadFromStream where the stream source is the node of an xml DOM.
- teeChart is populated with data and possibly default options are overridden. This occurs in ASP on the server.
- the populated teechart is exported to a file on the Web server via export.SaveToFile
- Browser page is created

Client side:
- teeChart is created
- XMLHttp async call downloads chart bytes from file on web server
- teeChart.import.LoadFromStream is used to load those bytes to the chart

I will look into creating a small test case to reproduce the problem.

>>>>>
When using LoadFromURL are you running against a repository of existing tee files or are those files generated on the fly at call time?
>>>>>
Generated on the fly.

>>>>>
Does the application hang whilst the load is running or have you not yet been able to identify the exact failure moment?
>>>>>
It's not exactly clear when the hang occurs.

>>>>>
If you have a mockup example or code that could help us rebuild/reproduce the application here, plus any excerpt from the error log please put it on the news://www.steema.net/steema.public.attachments newsgroup and we'll see if we can reproduce the error.
>>>>>
I was going to upload an IISState trace but I can't get to the news server. I think our company has blocked news.

>>>>>
What is your server platform (hardware/software)?
>>>>>
OS Name Microsoft(R) Windows(R) Server 2003, Standard Edition
Version 5.2.3790 Service Pack 1 Build 3790
System Manufacturer Dell Computer Corporation
System Model OptiPlex GX270
Processor x86 Family 15 Model 2 Stepping 9 GenuineIntel ~2793 Mhz
Total Physical Memory 1,021.95 MB

Hello,

We've been testing different load situations in IIS and can reproduce an intermittent failure when running TeeChart under heavy load with multiple concurrent accesses. As yet I have no specific details on cause/solution but we're looking into it and will post any update here.

Regards,
Marc Meumann

That's great news. On this end we have not been able to reproduce this in our development environment. Our QA team has been experiencing the hang on average about once a week. They have been running IISState -p pid_for_w3wp and have consistenly turned up a stack trace like the following for the thread that all the other threads are waiting on.

Thread ID: 36
System Thread ID: f7c
Kernel Time: 0:1:24.703
User Time: 0:0:21.421
*** WARNING: Unable to verify checksum for C:\WINDOWS\system32\TeeChart6.ocx
*** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\WINDOWS\system32\TeeChart6.ocx -
Thread Type: Idle ASP thread
# ChildEBP RetAddr
00 05d3f9b0 7739ca95 ntdll!KiFastSystemCallRet
01 05d3f9ec 773ad01a USER32!NtUserMessageCall+0xc
02 05d3fa0c 504dc8cb USER32!SendMessageA+0x7f
WARNING: Stack unwind information not available. Following frames may be wrong.
03 05d3fa30 504242fc TeeChart6!DllUnregisterServer+0x9eb9f
04 05d3fa50 5042464e TeeChart6+0x42fc
05 05d3fab0 7c82257a TeeChart6+0x464e
06 05d3fad0 7c818145 ntdll!LdrpCallInitRoutine+0x14
07 05d3fbe4 77e67b95 ntdll!LdrUnloadDll+0x40e
08 05d3fbf8 776b46fb kernel32!FreeLibrary+0x41
09 05d3fc04 776b470f ole32!CClassCache::CDllPathEntry::CFinishObject::Finish+0x2f
0a 05d3fc18 776b4c33 ole32!CClassCache::CFinishComposite::Finish+0x1d
0b 05d3fd7c 776b4b89 ole32!CClassCache::FreeUnused+0x1bc
0c 05d3fd8c 776b5137 ole32!CoFreeUnusedLibrariesEx+0x36
0d 05d3fd98 75bd78e3 ole32!CoFreeUnusedLibraries+0x9
0e 05d3fd9c 7739c3b7 comsvcs!STAFreeLibTimerProc+0x6
0f 05d3fdc8 77393147 USER32!InternalCallWinProc+0x28
10 05d3fe40 7739ce45 USER32!UserCallWinProc+0xf4
11 05d3fea0 7739c778 USER32!DispatchMessageWorker+0x138
12 05d3feb0 75bd7a78 USER32!DispatchMessageW+0xf
13 05d3fecc 75bd79a5 comsvcs!CSTAQueueLessMessageWork::DoWork+0x4e
14 05d3fee4 75bd833e comsvcs!CSTAThread::DoWork+0x18
15 05d3ff04 75bd878a comsvcs!CSTAThread::ProcessQueueWork+0x37
16 05d3ff84 77bcb530 comsvcs!CSTAThread::WorkerLoop+0x17c
17 05d3ffb8 77e66063 msvcrt!_endthreadex+0xa3
18 05d3ffec 00000000 kernel32!BaseThreadStart+0x34

Because of the warning that the stack frames starting with teeChart may be wrong we have been trying to verify what IISState is showing us. We've asked QA to add the -d parameter to IISState to produce a dump. For a variety of reasons they have yet to capture a valid dump for the w3wp process. Lately I installed DebugDiag and established a hang rule. I'm hoping to gather some more information from this tool. See:
http://www.microsoft.com/downloads/deta ... layLang=en

So right now we are waiting on the next hang.

Hello,

To keep you updated, we've run tests across several machines with different processor setups and are finalising a modification that has been made to TeeChart to improve resilience in IIS and multi-threaded applications. Load tests have shown that resilience problems can occur, under extreme loads and particularly applicable to hyper-threaded machines.

Without knowing the full information of your server environment I'm not sure that the modifications will have direct relevance to the issue that is occurring at your server but we will make versions available for testing within the next few days so that you can evaluate performance.

Version revisions will be made available for TeeChart AX v6 and TeeChart AX v7.

Regards,
Marc Meumann

Great news. We discovered that our testing group had active virus scanning on their Web machine. They have been running with that off and we haven't had a hang in 22 days. Previously we were having a hang on average one out of every five days. It will be interesting to see if we can run your new version with active virus scanning. Do you ever recommend to customers running in a Web server environment to deactivate virus scanning?

Hello,

We don't have experience of problems related to virus scan software in known TeeChart installation environments. Problems could occur perhaps, if a virus scanner were to take control an area where temporary files were being generated, but it's something that would need checking according to the way a specific virues checker works.

Regards,
Marc Meumann

Hello,

Status update:
We're going to hold on to the update versions for a few more days. There are still tests left to run as there are some parallel modifications we'd like to check, but we'll post the updates as soon as possible, sometime over the next 10 days.

Regards,
Marc Meumann

In a post on 3/31 you were finalizing a fix to "improve resilience in IIS and mutli-threaded applications" On April 7 this fix was going to be available within 10 days. Is this fix in teeChart6.ocx 6.0.0.6?
Thanks

Hello,

We were not satisfied with some of the test results due to possible implications to different application types. We held onto the version pending further feedback as it seemed that some problems in this area may have been provoked by environmental circumstances (ie possible influence of virus scanner).

There are still some minor inner improvements that could however be applied in version 6. We'll program an update to be posted.

Regards,
Marc Meumann

Marc, thanks for your reply. a couple of questions:
- Will you make a posting here when the update is ready?
- Where will the update be posted?

- You mentioned that "some problems in this area may have been provoked by environmental circumstances (i.e. possible influence of virus scanner)." From our testing I would say that is definitely true. I would say that virus scanners make the problem worse, but running without a virus scanner does not necessarily eliminate the problem. Are you implying that you don't support running in a Web environment (i.e. multithreaded, multiuser) with an active virus scanner?

Thanks.