State is what networking people call memory. One of the important design principles in the Internet has always been to minimise the number of places that need to keep track of who is doing what.
In the case of stateless information servers this means that they do not keep track of which clients are accessing them. In other words, between one access and the next, the server and protocol are constructed in such a way that they do not care who, why, how, when or where the next access comes from.
This is essential to the reliability of the server, and to making such systems work in very large scale networks such as the Internet with potentially huge numbers of clients: if the server did depend on a client, then any client failure, or network failure would leave the server in the lurch, possibly not able to continue, or else serving other clients with reduced resources.
Another use of the term stateless is to describe whether or not the server keeps note of the actual data from each access by a client (irrespective of whether it notes who the client was). This is called server caching.
Server Caching is a way of improving the response time of a server. Usually, servers keep data on disk. If they keep a copy of all the most frequently or most recently accessed data in memory, they may be able to respond to new (or repeating) clients more quickly.