From time to time my cloud controller rejects all connections. No matter what I do (listing instances with euca-describe-instances, logging to web front-end) I always get error 403 (forbidden). I have tried different users. After restarting eucalyptus-cloud everything backs to normal. I have Debian Squeeze with eucalyptus 2.0.1 from your debian's repository.
Processing Administration::InstancesController#index (for 84.10.109.25 at 2011-05-09 08:14:33) [GET
]
New RightAws::Ec2 using shared connections mode
Opening new HTTP connection to 192.168.0.1:8773
##### RightAws::Ec2 returned an error: 403 Forbidden
Failure: 403 Forbidden
javax.persistence.PersistenceException: org.hibernate.exception.JDBCConnectionException: Cannot ope
n connection
#####
##### RightAws::Ec2 request: http://192.168.0.1:8773/services/Eucalyptus?AWSAccessKeyId=uhaZbw3dR\n0Tjq7abcdecSmrdAGcuTjuXL1J1Q&Action=DescribeInstances&SignatureMethod=HmacSHA256&SignatureVersion=2
&Timestamp=2011-05-09T06%3A14%3A33.000Z&Version=2010-08-31&Signature=yDcYjKKZMgYIJBvkbIec7FK1HH5Fw4
zyH6%2FN6mcJDrg%3D ####
Closing HTTP connection to 192.168.0.1:8773, reason: 'RightAws::AWSErrorHandler: code: 403: 'Forb
idden''
Got RightAws::AwsError, http_code is: '403'
RightAws::AwsError (403: Forbidden (http://192.168.0.1:8773/services/Eucalyptus?AWSAccessKeyId=uh
aZbw3dR0Tjq7xX1aZcSmrdAGcuTjuXL1J1Q&Action=DescribeInstances&SignatureMethod=HmacSHA256&SignatureVe
rsion=2&Timestamp=2011-05-09T06%3A14%3A33.000Z&Version=2010-08-31&Signature=yDcYjKKZMgYIJBvkbIec7FK
1HH5Fw4zyH6%2FN6mcJDrg%3D)):
/web/winnie/production/shared/bundle/ruby/1.8/gems/right_aws-2.1.0/lib/awsbase/right_awsbase.rb:5
45:in `request_info_impl'
/web/winnie/production/shared/bundle/ruby/1.8/gems/right_aws-2.1.0/lib/ec2/right_ec2.rb:140:in `r
equest_info'
/web/winnie/production/shared/bundle/ruby/1.8/gems/right_aws-2.1.0/lib/awsbase/right_awsbase.rb:5
59:in `request_cache_or_info'
/web/winnie/production/shared/bundle/ruby/1.8/gems/right_aws-2.1.0/lib/ec2/right_ec2.rb:151:in `d
escribe_resources_with_list_and_options'
/web/winnie/production/shared/bundle/ruby/1.8/gems/right_aws-2.1.0/lib/ec2/right_ec2_instances.rb
:103:in `describe_instances'
app/models/eucalyptus.rb:28:in `instances'
app/models/eucalyptus.rb:65:in `rescue_timeout'
app/models/eucalyptus.rb:63:in `times'
app/models/eucalyptus.rb:63:in `rescue_timeout'
app/models/eucalyptus.rb:27:in `instances'
app/models/instance.rb:48:in `synchronize!'
app/controllers/administration/instances_controller.rb:33:in `synchronize_instances'
compass (0.11.1) lib/compass/app_integration/rails/actionpack2/action_controller.rb:7:in `process
'
warden (0.10.7) lib/warden/manager.rb:35:in `call'
warden (0.10.7) lib/warden/manager.rb:34:in `catch'
warden (0.10.7) lib/warden/manager.rb:34:in `call'
sass (3.1.1) rails/./lib/sass/plugin/rack.rb:54:in `call'
devise (1.0.11) lib/devise/rails/warden_compat.rb:43:in `call'
/web/winnie/production/shared/bundle/ruby/1.8/gems/hoptoad_notifier-2.4.9/lib/hoptoad_notifier/rack.rb:27:in `call'
/web/winnie/production/shared/bundle/ruby/1.8/gems/hoptoad_notifier-2.4.9/lib/hoptoad_notifier/user_informer.rb:12:in `call'
/var/lib/gems/1.8/gems/passenger-2.2.11/lib/phusion_passenger/rack/request_handler.rb:92:in `process_request'
/var/lib/gems/1.8/gems/passenger-2.2.11/lib/phusion_passenger/abstract_request_handler.rb:207:in `main_loop'
/var/lib/gems/1.8/gems/passenger-2.2.11/lib/phusion_passenger/railz/application_spawner.rb:418:in `start_request_handler'
/var/lib/gems/1.8/gems/passenger-2.2.11/lib/phusion_passenger/railz/application_spawner.rb:358:in `handle_spawn_application'
/var/lib/gems/1.8/gems/passenger-2.2.11/lib/phusion_passenger/utils.rb:184:in `safe_fork'
/var/lib/gems/1.8/gems/passenger-2.2.11/lib/phusion_passenger/railz/application_spawner.rb:354:in `handle_spawn_application'
/var/lib/gems/1.8/gems/passenger-2.2.11/lib/phusion_passenger/abstract_server.rb:352:in `__send__'
/var/lib/gems/1.8/gems/passenger-2.2.11/lib/phusion_passenger/abstract_server.rb:352:in `main_loop'
/var/lib/gems/1.8/gems/passenger-2.2.11/lib/phusion_passenger/abstract_server.rb:196:in `start_synchronously'
/var/lib/gems/1.8/gems/passenger-2.2.11/lib/phusion_passenger/abstract_server.rb:163:in `start'
/var/lib/gems/1.8/gems/passenger-2.2.11/lib/phusion_passenger/railz/application_spawner.rb:213:in `start'
/var/lib/gems/1.8/gems/passenger-2.2.11/lib/phusion_passenger/spawn_manager.rb:262:in `spawn_rails_application'
/var/lib/gems/1.8/gems/passenger-2.2.11/lib/phusion_passenger/abstract_server_collection.rb:126:in `lookup_or_add'
/var/lib/gems/1.8/gems/passenger-2.2.11/lib/phusion_passenger/spawn_manager.rb:256:in `spawn_rails_application'
/var/lib/gems/1.8/gems/passenger-2.2.11/lib/phusion_passenger/abstract_server_collection.rb:80:in `synchronize'
/var/lib/gems/1.8/gems/passenger-2.2.11/lib/phusion_passenger/abstract_server_collection.rb:79:in `synchronize'
/var/lib/gems/1.8/gems/passenger-2.2.11/lib/phusion_passenger/spawn_manager.rb:255:in `spawn_rails_application'
/var/lib/gems/1.8/gems/passenger-2.2.11/lib/phusion_passenger/spawn_manager.rb:154:in `spawn_application'
/var/lib/gems/1.8/gems/passenger-2.2.11/lib/phusion_passenger/spawn_manager.rb:287:in `handle_spawn_application'
/var/lib/gems/1.8/gems/passenger-2.2.11/lib/phusion_passenger/abstract_server.rb:352:in `__send__'
/var/lib/gems/1.8/gems/passenger-2.2.11/lib/phusion_passenger/abstract_server.rb:352:in `main_loop'
/var/lib/gems/1.8/gems/passenger-2.2.11/lib/phusion_passenger/abstract_server.rb:196:in `start_synchronously'
** [Hoptoad] Success: Net::HTTPOK
** [Hoptoad] Environment Info: [Ruby: 1.8.7] [Rails: 2.3.11] [Env: production]
** [Hoptoad] Response from Hoptoad:
<?xml version="1.0" encoding="UTF-8"?>
<notice>
<error-id type="integer">6510333</error-id>
<url>http://winnie.hoptoadapp.com/errors/6510333/notices/1087938493</url>
<id type="integer">1087938493</id>
</notice>
Rendering /web/winnie/production/releases/20110506075758/public/500.html (500 Internal Server Error)
eucalyptus logs:
08:27:28 DEBUG [Context:New I/O server worker #1-13] :1304922448929:Context:CONTEXT_CREATE:7eeb595a-176d-4e6f-9104-1026f7c6c37b:[id: 0x6177d16c, /192.168.0.1:47633 => /192.168.0.1:8773]:NioHttpDecoder.decode.129
08:27:28 TRACE [NioServerHandler:New I/O server worker #1-13]
08:27:28 TRACE [EucalyptusQueryPipeline:New I/O server worker #1-13] :1304922448931:EucalyptusQueryPipeline:PIPELINE_UNROLL:EucalyptusQueryPipeline:PipelineRegistry.find.103
08:27:28 TRACE [InternalQueryPipeline:New I/O server worker #1-13] :1304922448932:InternalQueryPipeline:PIPELINE_UNROLL:InternalQueryPipeline:PipelineRegistry.find.103
08:27:28 TRACE [PipelineRegistry:New I/O server worker #1-13] :1304922448933:PipelineRegistry:PIPELINE_DUPLICATE:internal-query-pipeline-Eucalyptus:InternalQueryPipeline:NioServerHandler.lookupPipeline.137
08:27:28 TRACE [EucalyptusQueryPipeline:New I/O server worker #1-13] :1304922448934:EucalyptusQueryPipeline:PIPELINE_HANDLER:decoder:NioHttpDecoder:NioServerHandler.lookupPipeline.138
08:27:28 TRACE [EucalyptusQueryPipeline:New I/O server worker #1-13] :1304922448935:EucalyptusQueryPipeline:PIPELINE_HANDLER:encoder:HttpResponseEncoder:NioServerHandler.lookupPipeline.138
08:27:28 TRACE [EucalyptusQueryPipeline:New I/O server worker #1-13] :1304922448936:EucalyptusQueryPipeline:PIPELINE_HANDLER:chunkedWriter:ChunkedWriteHandler:NioServerHandler.lookupPipeline.138
08:27:28 TRACE [EucalyptusQueryPipeline:New I/O server worker #1-13] :1304922448937:EucalyptusQueryPipeline:PIPELINE_HANDLER:handler:NioServerHandler:NioServerHandler.lookupPipeline.138
08:27:29 TRACE [EucalyptusQueryPipeline:New I/O server worker #1-13] :1304922448938:EucalyptusQueryPipeline:PIPELINE_HANDLER:hmac-v2-verify:HmacHandler:NioServerHandler.lookupPipeline.138
08:27:29 TRACE [ReentrantListenerRegistry:SystemClockTimer] :1304922449650:ReentrantListenerRegistry:LISTENER_EVENT_FIRED:RemoteBootstrapperClient:com.eucalyptus.event.ClockTick@4b057dd5:ReentrantListenerRegistry.fireEvent.68
08:27:29 TRACE [EucalyptusQueryPipeline:New I/O server worker #1-13] :1304922449651:EucalyptusQueryPipeline:PIPELINE_HANDLER:timestamp-verify:QueryTimestampHandler:NioServerHandler.lookupPipeline.138
08:27:29 TRACE [ReentrantListenerRegistry:SystemClockTimer] :1304922449651:ReentrantListenerRegistry:LISTENER_EVENT_FIRED:LocalDatabaseBootstrapper:com.eucalyptus.event.ClockTick@4b057dd5:ReentrantListenerRegistry.fireEvent.68
08:27:29 TRACE [EucalyptusQueryPipeline:New I/O server worker #1-13] :1304922449652:EucalyptusQueryPipeline:PIPELINE_HANDLER:restful-binding:EucalyptusQueryBinding:NioServerHandler.lookupPipeline.138
08:27:29 TRACE [LocalDatabaseBootstrapper:SystemClockTimer] -> Ping database.
08:27:29 TRACE [EucalyptusQueryPipeline:New I/O server worker #1-13] :1304922449653:EucalyptusQueryPipeline:PIPELINE_HANDLER:service-sink:ServiceSinkHandler:NioServerHandler.lookupPipeline.138
08:27:29 TRACE [ReentrantListenerRegistry:SystemClockTimer] :1304922449654:ReentrantListenerRegistry:LISTENER_EVENT_FIRED:ClusterCertificateHandler:com.eucalyptus.event.ClockTick@4b057dd5:ReentrantListenerRegistry.fireEvent.68
08:27:29 TRACE [ReentrantListenerRegistry:SystemClockTimer] :1304922449655:ReentrantListenerRegistry:LISTENER_EVENT_FIRED:Dummy:com.eucalyptus.event.ClockTick@4b057dd5:ReentrantListenerRegistry.fireEvent.68
08:27:29 TRACE [TimedEvictionSet:New I/O server worker #1-13] Evicting previous entry:
4332710005527894
08:27:29 TRACE [ReentrantListenerRegistry:SystemClockTimer] :1304922449655:ReentrantListenerRegistry:LISTENER_EVENT_FIRED:NetworkStateHandler:com.eucalyptus.event.ClockTick@4b057dd5:ReentrantListenerRegistry.fireEvent.68
08:27:29 TRACE [ReentrantListenerRegistry:SystemClockTimer] :1304922449657:ReentrantListenerRegistry:LISTENER_EVENT_FIRED:LogStateHandler:com.eucalyptus.event.ClockTick@4b057dd5:ReentrantListenerRegistry.fireEvent.68
08:27:29 TRACE [ReentrantListenerRegistry:SystemClockTimer] :1304922449657:ReentrantListenerRegistry:LISTENER_EVENT_FIRED:ResourceStateHandler:com.eucalyptus.event.ClockTick@4b057dd5:ReentrantListenerRegistry.fireEvent.68
08:27:29 ERROR [TxHandle:New I/O server worker #1-13] javax.persistence.PersistenceException: org.hibernate.exception.JDBCConnectionException: Cannot open connection
javax.persistence.PersistenceException: org.hibernate.exception.JDBCConnectionException: Cannot open connection
at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1235)
at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1168)
at org.hibernate.ejb.AbstractEntityManagerImpl.throwPersistenceException(AbstractEntityManagerImpl.java:1245)
at org.hibernate.ejb.TransactionImpl.begin(TransactionImpl.java:63)
at com.eucalyptus.entities.TxHandle.<init>(TxHandle.java:42)
at com.eucalyptus.entities.EntityWrapper.<init>(EntityWrapper.java:109)
at com.eucalyptus.auth.Authentication.getEntityWrapper(Authentication.java:102)
at com.eucalyptus.auth.DatabaseAuthProvider.lookupQueryId(DatabaseAuthProvider.java:275)
at com.eucalyptus.auth.Users.lookupQueryId(Users.java:119)
at com.eucalyptus.auth.login.Hmacv2LoginModule.authenticate(Hmacv2LoginModule.java:32)
at com.eucalyptus.auth.login.Hmacv2LoginModule.authenticate(Hmacv2LoginModule.java:19)
at com.eucalyptus.auth.api.BaseLoginModule.login(BaseLoginModule.java:100)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:784)
at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
at javax.security.auth.login.LoginContext$5.run(LoginContext.java:721)
at javax.security.auth.login.LoginContext$5.run(LoginContext.java:719)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.login.LoginContext.invokeCreatorPriv(LoginContext.java:718)
at javax.security.auth.login.LoginContext.login(LoginContext.java:590)
at com.eucalyptus.ws.handlers.HmacHandler.incomingMessage(HmacHandler.java:116)
at com.eucalyptus.ws.handlers.MessageStackHandler.handleUpstream(MessageStackHandler.java:127)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:567)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:803)
at com.eucalyptus.ws.server.NioServerHandler.messageReceived(NioServerHandler.java:127)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:87)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:567)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:803)
at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:114)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:567)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:803)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:385)
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.unfoldAndfireMessageReceived(ReplayingDecoder.java:459)
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:443)
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:381)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:87)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:567)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:803)
at com.eucalyptus.ws.handlers.http.NioSslHandler.handleUpstream(NioSslHandler.java:31)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:567)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:562)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:342)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:329)
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:330)
at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:203)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:110)
at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:53)
at org.jboss.netty.handler.execution.MemoryAwareThreadPoolExecutor$MemoryAwareRunnable.run(MemoryAwareThreadPoolExecutor.java:502)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: org.hibernate.exception.JDBCConnectionException: Cannot open connection
at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:99)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:52)
at org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:449)
at org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:167)
at org.hibernate.jdbc.JDBCContext.connection(JDBCContext.java:142)
at org.hibernate.transaction.JDBCTransaction.begin(JDBCTransaction.java:85)
at org.hibernate.impl.SessionImpl.beginTransaction(SessionImpl.java:1463)
at org.hibernate.ejb.TransactionImpl.begin(TransactionImpl.java:60)
... 51 more
Caused by: java.sql.SQLException: socket creation error
at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
at org.hsqldb.jdbc.jdbcConnection.<init>(Unknown Source)
at org.hsqldb.jdbcDriver.getConnection(Unknown Source)
at org.hsqldb.jdbcDriver.connect(Unknown Source)
at java.sql.DriverManager.getConnection(DriverManager.java:620)
at java.sql.DriverManager.getConnection(DriverManager.java:169)
at org.logicalcobwebs.proxool.DefaultConnectionBuilder.buildConnection(DefaultConnectionBuilder.java:39)
at org.logicalcobwebs.proxool.Prototyper.buildConnection(Prototyper.java:159)
at org.logicalcobwebs.proxool.ConnectionPool.getConnection(ConnectionPool.java:211)
at org.logicalcobwebs.proxool.ProxoolDriver.connect(ProxoolDriver.java:89)
at java.sql.DriverManager.getConnection(DriverManager.java:620)
at java.sql.DriverManager.getConnection(DriverManager.java:222)
at org.hibernate.connection.ProxoolConnectionProvider.getConnection(ProxoolConnectionProvider.java:75)
at org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:446)
... 56 more
08:27:29 TRACE [ReentrantListenerRegistry:SystemClockTimer] :1304922449658:ReentrantListenerRegistry:LISTENER_EVENT_FIRED:VmStateHandler:com.eucalyptus.event.ClockTick@4b057dd5:ReentrantListenerRegistry.fireEvent.68
08:27:29 DEBUG [BaseLoginModule:New I/O server worker #1-13] java.lang.RuntimeException: javax.persistence.PersistenceException: org.hibernate.exception.JDBCConnectionException: Cannot open connection
java.lang.RuntimeException: javax.persistence.PersistenceException: org.hibernate.exception.JDBCConnectionException: Cannot open connection
at com.eucalyptus.entities.TxHandle.<init>(TxHandle.java:48)
at com.eucalyptus.entities.EntityWrapper.<init>(EntityWrapper.java:109)
at com.eucalyptus.auth.Authentication.getEntityWrapper(Authentication.java:102)
at com.eucalyptus.auth.DatabaseAuthProvider.lookupQueryId(DatabaseAuthProvider.java:275)
at com.eucalyptus.auth.Users.lookupQueryId(Users.java:119)
at com.eucalyptus.auth.login.Hmacv2LoginModule.authenticate(Hmacv2LoginModule.java:32)
at com.eucalyptus.auth.login.Hmacv2LoginModule.authenticate(Hmacv2LoginModule.java:19)
at com.eucalyptus.auth.api.BaseLoginModule.login(BaseLoginModule.java:100)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:784)
at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
at javax.security.auth.login.LoginContext$5.run(LoginContext.java:721)
at javax.security.auth.login.LoginContext$5.run(LoginContext.java:719)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.login.LoginContext.invokeCreatorPriv(LoginContext.java:718)
at javax.security.auth.login.LoginContext.login(LoginContext.java:590)
at com.eucalyptus.ws.handlers.HmacHandler.incomingMessage(HmacHandler.java:116)
at com.eucalyptus.ws.handlers.MessageStackHandler.handleUpstream(MessageStackHandler.java:127)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:567)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:803)
at com.eucalyptus.ws.server.NioServerHandler.messageReceived(NioServerHandler.java:127)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:87)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:567)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:803)
at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:114)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:567)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:803)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:385)
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.unfoldAndfireMessageReceived(ReplayingDecoder.java:459)
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:443)
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:381)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:87)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:567)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:803)
at com.eucalyptus.ws.handlers.http.NioSslHandler.handleUpstream(NioSslHandler.java:31)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:567)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:562)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:342)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:329)
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:330)
at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:203)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:110)
at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:53)
at org.jboss.netty.handler.execution.MemoryAwareThreadPoolExecutor$MemoryAwareRunnable.run(MemoryAwareThreadPoolExecutor.java:502)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: javax.persistence.PersistenceException: org.hibernate.exception.JDBCConnectionException: Cannot open connection
at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1235)
at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1168)
at org.hibernate.ejb.AbstractEntityManagerImpl.throwPersistenceException(AbstractEntityManagerImpl.java:1245)
at org.hibernate.ejb.TransactionImpl.begin(TransactionImpl.java:63)
at com.eucalyptus.entities.TxHandle.<init>(TxHandle.java:42)
... 50 more
Caused by: org.hibernate.exception.JDBCConnectionException: Cannot open connection
at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:99)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:52)
at org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:449)
at org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:167)
at org.hibernate.jdbc.JDBCContext.connection(JDBCContext.java:142)
at org.hibernate.transaction.JDBCTransaction.begin(JDBCTransaction.java:85)
at org.hibernate.impl.SessionImpl.beginTransaction(SessionImpl.java:1463)
at org.hibernate.ejb.TransactionImpl.begin(TransactionImpl.java:60)
... 51 more
Caused by: java.sql.SQLException: socket creation error
at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
at org.hsqldb.jdbc.jdbcConnection.<init>(Unknown Source)
at org.hsqldb.jdbcDriver.getConnection(Unknown Source)
at org.hsqldb.jdbcDriver.connect(Unknown Source)
at java.sql.DriverManager.getConnection(DriverManager.java:620)
at java.sql.DriverManager.getConnection(DriverManager.java:169)
at org.logicalcobwebs.proxool.DefaultConnectionBuilder.buildConnection(DefaultConnectionBuilder.java:39)
at org.logicalcobwebs.proxool.Prototyper.buildConnection(Prototyper.java:159)
at org.logicalcobwebs.proxool.ConnectionPool.getConnection(ConnectionPool.java:211)
at org.logicalcobwebs.proxool.ProxoolDriver.connect(ProxoolDriver.java:89)
at java.sql.DriverManager.getConnection(DriverManager.java:620)
at java.sql.DriverManager.getConnection(DriverManager.java:222)
at org.hibernate.connection.ProxoolConnectionProvider.getConnection(ProxoolConnectionProvider.java:75)
at org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:446)
... 56 more
08:27:29 TRACE [ReentrantListenerRegistry:SystemClockTimer] :1304922449661:ReentrantListenerRegistry:LISTENER_EVENT_FIRED:AddressStateHandler:com.eucalyptus.event.ClockTick@4b057dd5:ReentrantListenerRegistry.fireEvent.68
08:27:29 DEBUG [BaseLoginModule:New I/O server worker #1-13] Login aborted.
08:27:29 DEBUG [BaseLoginModule:New I/O server worker #1-13] Login aborted.
08:27:29 DEBUG [BaseLoginModule:New I/O server worker #1-13] Login aborted.
08:27:29 DEBUG [BaseLoginModule:New I/O server worker #1-13] Login aborted.
08:27:29 DEBUG [BaseLoginModule:New I/O server worker #1-13] Login aborted.
08:27:29 TRACE [ServiceSinkHandler:New I/O server worker #1-13] ServiceSinkHandler[incoming]: [id: 0x6177d16c, /192.168.0.1:47633 => /192.168.0.1:8773] EXCEPTION: javax.security.auth.login.LoginException: javax.persistence.PersistenceException: org.hibernate.exception.JDBCConnectionException: Cannot open connection
08:27:29 TRACE [ServiceSinkHandler:New I/O server worker #1-13] [id: 0x6177d16c, /192.168.0.1:47633 => /192.168.0.1:8773]
javax.security.auth.login.LoginException: javax.persistence.PersistenceException: org.hibernate.exception.JDBCConnectionException: Cannot open connection
at com.eucalyptus.auth.api.BaseLoginModule.login(BaseLoginModule.java:104)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:784)
at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
at javax.security.auth.login.LoginContext$5.run(LoginContext.java:721)
at javax.security.auth.login.LoginContext$5.run(LoginContext.java:719)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.login.LoginContext.invokeCreatorPriv(LoginContext.java:718)
at javax.security.auth.login.LoginContext.login(LoginContext.java:590)
at com.eucalyptus.ws.handlers.HmacHandler.incomingMessage(HmacHandler.java:116)
at com.eucalyptus.ws.handlers.MessageStackHandler.handleUpstream(MessageStackHandler.java:127)
at com.eucalyptus.ws.server.NioServerHandler.messageReceived(NioServerHandler.java:127)
at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:114)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:385)
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.unfoldAndfireMessageReceived(ReplayingDecoder.java:459)
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:443)
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:381)
at com.eucalyptus.ws.handlers.http.NioSslHandler.handleUpstream(NioSslHandler.java:31)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:342)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:329)
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:330)
at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:203)
at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:53)
at org.jboss.netty.handler.execution.MemoryAwareThreadPoolExecutor$MemoryAwareRunnable.run(MemoryAwareThreadPoolExecutor.java:502)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
08:27:29 TRACE [ServiceSinkHandler:New I/O server worker #1-13] ServiceSinkHandler[outgoing]: class org.jboss.netty.channel.DownstreamChannelStateEvent
08:27:29 TRACE [ServiceSinkHandler:New I/O server worker #1-13] ServiceSinkHandler[incoming]: [id: 0x6177d16c] DISCONNECTED
08:27:29 TRACE [ServiceSinkHandler:New I/O server worker #1-13] ServiceSinkHandler[incoming]: [id: 0x6177d16c] UNBOUND
08:27:29 TRACE [ServiceSinkHandler:New I/O server worker #1-13] ServiceSinkHandler[incoming]: [id: 0x6177d16c] CLOSED
08:27:29 TRACE [ServiceSinkHandler:New I/O server worker #1-13] ServiceSinkHandler[incoming]: [id: 0x6177d16c] WRITTEN_AMOUNT: 206
08:27:30 INFO [DescribePublicAddressesType:New I/O client worker #2-19] :1304922450714:DescribePublicAddressesType:MSG_SENT:[DescribePublicAddressesType correlationId=b15294c6-4474-4ded-aab5-a050a6c04926 userId=eucalyptus effectiveUserId=eucalyptus _return=true statusMessage=null]:DefaultChannelFuture.notifyListener.326
08:27:30 INFO [DescribePublicAddressesResponseType:New I/O client worker #2-19] :1304922450733:DescribePublicAddressesResponseType:MSG_SENT:[DescribePublicAddressesResponseType addresses=[] mapping=[] correlationId=null userId=eucalyptus effectiveUserId=null _return=false statusMessage=NOTSUPPORTED]:SimpleChannelHandler.handleUpstream.105
08:27:30 TRACE [ChannelStateMonitor:New I/O client worker #2-19] :1304922450736:AddressStateHandler:SOCKET_CLOSE:21:/192.168.0.1:33359:/192.168.0.1:8774:ChannelStateMonitor.channelClosed.40
08:27:30 TRACE [ChannelStateMonitor:New I/O client worker #2-19] :1304922450737:AddressStateHandler:SOCKET_BYTES_READ:5537:237.61905:ChannelStateMonitor.channelClosed.40
08:27:30 TRACE [ChannelStateMonitor:New I/O client worker #2-19] :1304922450738:AddressStateHandler:SOCKET_BYTES_WRITE:4990:237.61905:ChannelStateMonitor.channelClosed.40
This issue is now being tracked upstream at http://
Please watch that issue for further updates.