Please Login or Register.

Network Issues

Affecting Server Cloudtrax - Priority - High (Resolved)

Affecting System - Cloudtrax | Priority - Critical

Affecting System - | Priority - High

Users may experience difficulties logging in to the Cloudtrax dashboard. We are currently investigating this issue, and hope to have a resolution shortly.

Update: As of 8:50am Users should now be able to log-in to Cloudtrax.

Date - 10/21/2013 08:30 - 10/21/2013 08:50
Last Updated - 10/21/2013 09:05

Affecting System: Cloudtrax | Priority - Medi (Resolved)

Affecting Other - Paypal | Priority - Medium

Paypal purchases currently do not issue the voucher code. We are currently investigating this issue, and hope to have a fix by the end of the day.

Date - 09/03/2013 12:11
Last Updated - 09/03/2013 21:24

Affecting System: Cloudtrax | Priority - High (Resolved)

Affecting System - | Priority - High


Cloudtrax Lobby Assistant and the Voucher Creation system is now functioning as intended. In addition, a minor issue regarding splash page images has been resolved.


Cloudtrax Lobby Assistant and the Voucher Creation system are currently unreachable - we're aware of the outage and are currently working on a resolution.

We will provide updates to this notice once we know more.

Date - 08/09/2013 08:24
Last Updated - 08/09/2013 10:18

Cloudtrax Status (Resolved)

Affecting System - Cloudtrax | Priority - High

This weekend there was an event that caused some nodes to lose connectivity to Cloudtrax. 

The Cloudtrax connectivity issue affects some gateways on master logins beginning with the letters A-C (and some D) only.

While some or all nodes appear down, the network is actually functioning.  The only functions known to be affected are:

1.  Checkin

2.  Vouchers

We have implemented a fix to recover as many nodes as possible without needing a power cycle.  This recovers the above functionality on all nodes that can see at least one gateway that is checking in normally.

At least one gateway in each node mesh cluster (nodes that can see each other via mesh) needs to be up to restore checkins to all nodes in that mesh cluster.  If all your nodes are in a larger mesh, power cycling a single gateway in your network should be enough to restore them all.  If they are in isolated clusters, one gateway should be power cycled per cluster to restore that cluster.  Regardless, repeaters will not need anything:  Once their gateway is working, they will be able to check in again.

We deeply apologize for the inconvenience.

Date - 05/19/2013 20:27
Last Updated - 06/02/2013 21:49

Cloudtrax Server Unreachable (Resolved)

Affecting System - Cloudtrax | Priority - High

**UPDATE** 20:00 11/18/2012

The cloudtrax service has been restored

The cloudtrax server was down due to a problem with Amazon Web Services.  Cloudtrax was not reachable between 16:30 and 20:00 Pacific Standard Time - you'll see this reflected in the 24 hour outage graphs.

Networks that utilize vouchers and require vouchers for access would have had difficulty accessing the internet.  All other networks not utilizing vouchers would have seen no service interruptions.

Sorry for the inconvenience

Open-Mesh Team


Cloudtrax is currently unreachable - we're aware of the outage and are currently working on getting it back up.


We'll update this notice once we know more.

Date - 11/18/2012 16:42 - 11/22/2012 12:49
Last Updated - 11/18/2012 21:56

Cloudtrax server unreachable (Resolved)

Affecting System - Cloudtrax | Priority - Critical



The EC2 team got back with us, as of 7:09AM PDT: "AWS Console fix has been deployed and customers should now be able to stop instances that are stuck in pending. This will only work for instances that were stopped previously. New instances should rather be terminated."

Please note that root cause investigation is still ongoing and will be posted publicly when available. We are very aware that many customers are effected and apologize for the impact on your business. If you have any concerns or cloud architecture questions, please don't hesitate to reply or open a new case at

Best regards,

Joshua F.
Amazon Web Services



The cloudtrax controller has been restored.  Because of the massive power outage at Amazon AWS EC2, data on the hard disk was corrupted.  We were forced to restore to a backup image that was taken 36 hours ago.

This means that any data entered into cloudtrax over the last 36 hours has been lost and will need to be re-entered.  This includes usage statistics, cloudtrax settings, and generated vouchers.

Once again, we apologize for the outage, and thank you for your patience and understanding while we worked to restore the controller.


Open-Mesh Team



Despite the report from Amazon that all services are back to normal, we are still struggling to bring the server up.  We've received no help from Amazon with the problem.  We do not have an ETA yet.

We apologize and have been at this non-stop since the outage at 8:50 PST.  We will keep you updated once we have more information.



Cloudtrax is currently down due to a problem at the data center where the server is hosted.  We are closely monitoring it and hoping for a quick resolution.


All non-voucher networks will not be affected by the outage.  Non-voucher Networks will continue to operate normally. 


Unfortunately voucher networks will not be accessible until cloudtrax is back up.


The outage has already made the news:


OpenMesh Team


Date - 06/14/2012 21:59
Last Updated - 06/15/2012 12:37

Cloudtrax Server Upgrade (Resolved)

Affecting System - Cloudtrax | Priority - Medium

The cloudtrax server is undergoing an upgrade.  The outage will be brief - approximately 5 - 10 minutes.


Date - 02/02/2012 09:24 - 02/02/2012 09:49
Last Updated - 02/02/2012 09:25

Cloudtrax outage (Resolved)

Affecting System - | Priority - Medium

Cloudtrax was not responsive between 17:00 and 19:00 PST - outages shown in cloudtrax during this time do not reflect actual network outages.

Date - 11/24/2011 17:13 - 11/28/2011 08:59
Last Updated - 11/24/2011 19:35

NG firmware users can't browse (Resolved)

Affecting System - | Priority - Critical

A bug was discovered on Sunday in a specific cloud controller reply (since corrected) that can cause NG nodes to stop handling DNS properly.  This can result in "0x02" errors (second line of the "uptime" column in the node list on the cloud controller), missing "speed test" results on repeaters, low memory (<6MB) and checkin problems.

The problem occurred Sunday morning pacific time.

In all cases where this bug happens, DNS is not functioning and users will not be able to browse.

Power cycling the nodes will correct the problem.  No reflashing or other measures are needed.  It is not possible to correct the issue remotely as the nodes stop processing dashboard replies even though they check in.

If your nodes are in wallplug enclosures (or other difficult to get to situations), it is possible to reset them via the AC breaker panel on site.  The nodes need the power off for about 1 second to come back up with the issue corrected.

Protection against this and similar issues will be included in the next NG firmware build. 

We apologize for this inconvenience.  If you have further questions, please email




Date - 10/23/2011 18:07 - 11/07/2011 08:42
Last Updated - 10/23/2011 19:05

24 Hour outage graphs (Resolved)

Affecting System - Cloudtrax | Priority - Medium

24-hour outages graphs out from 10:11am to 11:11am PST Oct 17.  Outages shown during this time do not reflect actual node outages on your network.

Date - 10/17/2011 10:11 - 10/17/2011 14:43
Last Updated - 10/17/2011 12:51

« Prev Page   Next Page »