Page 1 of 1

Push requests hanging causes high Apache thread use

Posted: 25 Jul 2014, 18:06
by felipe.gdr
When my app is running remotely behind an Apache Server several requests to atmosphere channels keep hanging in the "W" Sending Reply state.

So a few minutes after the app has been deployed I get an overload in the available request threads and apache stops serving my app.

The image bellow shows some of those requests as viewed in the /server-status monitoring page:

Image
Server uptime: 3 hours 22 minutes 24 seconds
Total accesses: 2511 - Total Traffic: 29.4 MB
.207 requests/sec - 2537 B/second - 12.0 kB/request
124 requests currently being processed, 126 idle workers
The total requests currently being processed number keeps increasing (it never gets smaller), and once it achieves the maximum (250) apache stops working....

I'm using Primefaces 3.5.14 with Atmosphere 1.0.8. My web server is JBoss 7.1.1.

This is my servlet config in web.xml

Code: Select all

	<servlet>
		<servlet-name>Push Servlet</servlet-name>
		<servlet-class>org.primefaces.push.PushServlet</servlet-class>
		<load-on-startup>1</load-on-startup>
		<async-supported>true</async-supported>
		<init-param>
			<param-name>org.atmosphere.websocket.maxTextMessageSize</param-name>
			<param-value>0</param-value>
		</init-param>
		<init-param>
			<param-name>org.atmosphere.useWebSocket</param-name>
			<param-value>true</param-value>
		</init-param>
		<init-param>
			<param-name>org.atmosphere.cpr.sessionSupport</param-name>
			<param-value>true</param-value>
		</init-param>
		<init-param>
			<param-name>org.atmosphere.useNative</param-name>
			<param-value>true</param-value>
		</init-param>
		<init-param>
			<param-name>org.atmosphere.resumeOnBroadcast</param-name>
			<param-value>true</param-value>
		</init-param>
		<init-param>
			<param-name>org.atmosphere.cpr.broadcasterCacheClass</param-name>
			<param-value>org.atmosphere.cache.HeaderBroadcasterCache</param-value>
		</init-param>
		<init-param>
			<param-name>org.atmosphere.cpr.broadcastFilterClasses</param-name>
			<param-value>org.atmosphere.client.TrackMessageSizeFilter</param-value>
		</init-param>
		<init-param>
			<param-name>org.atmosphere.useBlocking</param-name>
			<param-value>true</param-value>
		</init-param>
	</servlet>

Re: Push requests hanging causes high Apache thread use

Posted: 25 Jul 2014, 19:01
by kukeltje
are you using apache as a reverse proxy? Do you use the mod_websocket or similar? Otherwise apache cannot reverse proxy the requests afaik.

Re: Push requests hanging causes high Apache thread use

Posted: 25 Jul 2014, 21:05
by felipe.gdr
I'm using mod_jk

Re: Push requests hanging causes high Apache thread use

Posted: 25 Jul 2014, 21:52
by kukeltje

Re: Push requests hanging causes high Apache thread use

Posted: 25 Jul 2014, 21:59
by felipe.gdr
I have changed to mod_proxy, and now the used thread number no longer increases indefinitely, however it takes a few minutes for the threads to change from the "W" Sending Reply status to the "_" Waiting for Connection, so I'm worried that, once the application goes into production and I have maybe hundreds of simultaneous users, I'll run out of Apache threads...

I have some further input that may be useful to understand my scenario:

- every single page in my app has a socket (<p:socket>) component that listens to a certain channel. This socket is waiting for new tasks that might be created for the logged user.
- when I navigate to any page, a new request is sent with the a URL that looks like this: GET /myapp-web/primepush/tasks/55?X-Atmosphere-tracking-i. So a new thread is created in Apache with the status "W" Sending Reply.
- if I navigate away to a different page, another thread is created, but the previous one keeps hanging in the "W" Sending Reply for a little while.

Is this the expected behaviour?

Re: Push requests hanging causes high Apache thread use

Posted: 25 Jul 2014, 22:02
by felipe.gdr
Thanks for the link kukeltje.

But since I'm using Jboss 7.1.1 websockets won't work, right? At least my javascript console messages tell me that Comet is being used instead

Re: Push requests hanging causes high Apache thread use

Posted: 28 Jul 2014, 21:53
by kukeltje
No idea. Maybe ask onthe atmosphere list