Spring Boot application freezes after a day

0 votes

I have a Spring Boot application running on two servers. It accepts file upload requests (approx 1 KB) via Controller from 1000s of IOT devices.

Additionally there is a TCP Listener which also accepts GPS data as a Stream from these IOT devices.

I have setup the Tomcat max thread count to 3000 on one server and 5000 on a larger VM.

After a day or two my Spring Boot application freezes, i.e. it stops responding to any further API calls.

One reason, I suspect is that the IOT devices are not closing the TCP socket connection after sending the data.

Where could I be going wrong? I am running out of threads to handle new incoming requests?

############################

EDIT 1:

After investigating the thread dump and adding logs during during Socket connection and closure, I have discovered that the IOT device isn't closing connections. Instead the device is creating a new connection every time it wants to send new data without closing the previous one.

Before the app froze, 22500 open connections were automatically closed.

How can I handle this behavior? Should I close the connections forcefully after a certain timeout?

Jul 26, 2018 in IoT (Internet of Things) by Matt
• 2,260 points
502 views

1 answer to this question.

0 votes

You can mitigate the idle connections problem by hiding your service behind an Nginx or HAProxy which close idle connections by default after a given timeout.

Another option is to set server.connection-timeout in application properties (see server properties), but this appears to be applicable to HTTP only.

answered Jul 26, 2018 by anonymous2
• 4,260 points

Related Questions In IoT (Internet of Things)

0 votes
1 answer
0 votes
1 answer
0 votes
1 answer

How to prevent an application from publishing to a topic?

Devices and applications operate in a topic ...READ MORE

answered Jan 18 in IoT (Internet of Things) by Shubham
• 13,210 points
13 views
0 votes
1 answer

Downloading source code of Android Things

No because AndroidThings is still in preview ...READ MORE

answered Jul 5, 2018 in IoT (Internet of Things) by anonymous2
• 4,260 points
401 views
0 votes
1 answer

Creating Android Things' bundle and pushing it over OTA

Here are all the steps necessary to ...READ MORE

answered Jul 6, 2018 in IoT (Internet of Things) by anonymous2
• 4,260 points
36 views
0 votes
1 answer

AWS IoT - Access shadow through .Net, REST, with certificate

If you want to publish and/or subscribe ...READ MORE

answered Jul 25, 2018 in IoT (Internet of Things) by anonymous2
• 4,260 points
61 views
0 votes
1 answer

Android AllJoyn: Connection with second machine gives error of BusAttachement

Why dont you create two Interfaces, one ...READ MORE

answered Jul 25, 2018 in IoT (Internet of Things) by anonymous2
• 4,260 points
25 views
0 votes
1 answer

aws iot describe-endpoint::You must specify a region

This fixed the problem: Browse to https://console.aws.amazon.com/iam/home?region=us-west-2#security_credentialand then if ...READ MORE

answered Jul 11, 2018 in IoT (Internet of Things) by anonymous2
• 4,260 points
174 views
0 votes
1 answer

What is the maximum message length for a MQTT broker?

It's not entirely clear what you're asking ...READ MORE

answered Jul 17, 2018 in IoT (Internet of Things) by anonymous2
• 4,260 points
158 views