[Solved] Problems multicast ehcache

Archive Rancher 1.x
1

Hello everyone! I have a problem trying to configuring ehcache in a cluster environment. Let me explain more about the architecture of the application…

I have a java application running in a tomcat server. This application is wrapped in a docker image. This application works fine, and I run it using rancher and 2 machines, but the problem is that the application uses ehcache, and I’ve configured this using automatic mode. In automatic mode, ehcache uses a multicast address, and a multicast port. I’ve configured that, but the application could not replicate the cache.

Below are the configurations I used to run the application in rancher:

  1. First try: I’ve exposed the ports 8080 (tcp), 4001 (tcp, used to replicate the cache), and 4446 (udp. Used for the autodiscovery with the multicasts address). I’ve used “managed” network. => It didn’t work…

  2. Second try: I’ve exposed the ports 8080 (tcp), 4001 (tcp, used to replicate the cache), and 4446 (udp. Used for the autodiscovery with the multicasts address). I’ve used “host” network. => It didn’t work…

I’ve also tried exposing 4446 port with tcp, but it didn’t work neither.

Anyone can help me with this?

Thanks in advance!

2

I forgot to write that if I run the application in a non-docker environment (in two ubuntu servers), it works fine, and it replicates the cache without any problem,

Thanks again!

3

The overlay network is layer 3, not layer 2. No multicast.

4

But In the second try I used host network. It should work, right? Or what are I doing wrong?

5

Hey juandavidgc,

Shouldn’t that be TCP port 40001 rather than 4001? I believe this port is used with manual discovery and for the peer listener.

The 4446 port used for auto-discovery is also TCP, not UDP.

Info from here: http://www.ehcache.org/documentation/2.8/replication/rmi-replicated-caching.html.

As long as you have IP forwarding enabled (check with sysctl net.ipv4.ip_forward) multicast should work just fine in host networking mode. Of course, each container running ehcache MUST be on a different host, they can’t both listen to the same port.

If this doesn’t get you anywhere, please post your xml config.

6

Hello sjiveson. I’ve tested this but it didn’t work neither. Below are the files and the configuration in rancher.

Thanks!

ehcache.xml

<?xml version="1.0" encoding="UTF-8"?> 

<!--terracottaConfig url="localhost:9510" /-->

<!-- peer configuration -->

<!-- Listener port of this analyzer instance 8081 to put-->
<!-- <cacheManagerPeerProviderFactory
        class="net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory"
        properties="peerDiscovery=manual, rmiUrls="/>-->
<cacheManagerPeerProviderFactory class="net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory"
	properties="hostName=, peerDiscovery=automatic,multicastGroupAddress=230.0.0.1,multicastGroupPort=4446,timeToLive=64"/>

<!-- Listener port of this analyzer instance 8081 me put-->
<cacheManagerPeerListenerFactory
        class="net.sf.ehcache.distribution.RMICacheManagerPeerListenerFactory"
        properties="hostName=,port=40001,socketTimeoutMillis=2000"/>

<defaultCache eternal="true" maxElementsInMemory="500" overflowToDisk="false" memoryStoreEvictionPolicy="LFU">
    <bootstrapCacheLoaderFactory
            class="net.sf.ehcache.distribution.RMIBootstrapCacheLoaderFactory"
            properties="bootstrapAsynchronously=true, maximumChunkSizeBytes=5000000"
            propertySeparator=","/>
</defaultCache>

<cache name="identisiteCache" logging="true" eternal="false" timeToIdleSeconds="60" maxElementsInMemory="1000"
       overflowToDisk="false" statistics="true">
    <cacheEventListenerFactory class="net.sf.ehcache.distribution.RMICacheReplicatorFactory"
                               properties="replicateAsynchronously=false, replicatePuts=true, replicateUpdates=true,
  replicateUpdatesViaCopy=false, replicateRemovals=true "/>
</cache>

<cache name="utilConfigurationCache" logging="true" eternal="true" timeToLiveSeconds="0" timeToIdleSeconds="0"  maxElementsInMemory="500"
       overflowToDisk="false" statistics="true">
    <cacheEventListenerFactory class="net.sf.ehcache.distribution.RMICacheReplicatorFactory"
                               properties="replicateAsynchronously=false, replicatePuts=true, replicateUpdates=true,
  replicateUpdatesViaCopy=false, replicateRemovals=true "/>
</cache>
...
...

Dockerfile

RUN apt-get -y install software-properties-common
RUN apt-get -y install unzip

RUN
echo oracle-java7-installer shared/accepted-oracle-license-v1-1 select true | debconf-set-selections &&
add-apt-repository -y ppa:webupd8team/java &&
apt-get update &&
apt-get install -y oracle-java7-installer &&
rm -rf /var/lib/apt/lists/* &&
rm -rf /var/cache/oracle-jdk7-installer

ENV JAVA_HOME /usr/lib/jvm/java-7-oracle

RUN mkdir /opt/app

COPY app.zip /opt/app

WORKDIR /opt/app
RUN unzip app.zip

RUN rm -rf /opt/app/application/logs/catalina.out

RUN chmod -R a+x /opt/app/application/bin

RUN mkdir /opt/util
COPY PropertyReplacer.jar /opt/util/

WORKDIR /opt/app/application/bin/
ENV PATH /opt/app/application/bin:$PATH
EXPOSE 8080
EXPOSE 40001
EXPOSE 4446
COPY docker-entrypoint.sh /
RUN chmod a+x /docker-entrypoint.sh
ENTRYPOINT ["/docker-entrypoint.sh"]
CMD [“catalina.sh”, “run”]

Screenshot of the service configuration

screenshot.png1627×955 48.4 KB

7

Any idea why there’s reference to 8081 in your file?

Are you certain this is correctly commented out:

If so, can you enter whichever of these commands works and post back the output - this will confirm (or not) that the ports are being bound as we expect:

netstat -ltn
ss -ltn

Also, please confirm you only have single interface configured. This would have to use host mode networking to work so this relates to the host interfaces.

8

Thanks for your help! I solved this problem extending ehcache functionallity. I create another way to discover new members of the replica set using giddyup.

9

You’re welcome, glad you got it sorted as I was out of ideas.

10

@sjiveson I trapped in the same problem. My idea was also to create a new discovery (DNS or rancher metadata), then I found this post. Would you mind to share your solution?
Kind regards,
Michael.

11

Hey, I didn’t solve it, @juandavidgc did, perhaps he can share his solution?