low visibility of errors when using swift-init

Bug #639710 reported by clayg
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Object Storage (swift)
Fix Released
Low
clayg

Bug Description

When there's a typo in a config file, starting a server with swift-init can't display an error to stderr (already damonized) and can't log to syslog (not configed yet!)

It MAY be able to recognize that the server failed to start and possibly suggest you try to start it directly? (e.g. swift-init proxy start fails, try swift/bin/swift-proxy-server)

Related branches

clayg (clay-gerrard)
Changed in swift:
assignee: nobody → clayg (clay-gerrard)
importance: Undecided → Low
Revision history for this message
Mike Barton (redbo) wrote :

I think you could push the devnull dup2ing of the stdio fds down to where sys.stdout and stderror are replaced with LoggerFileObjects in common.(daemon|wsgi). Then any errors that occur between those two operations should go to the terminal instead of being blackholed.

Revision history for this message
clayg (clay-gerrard) wrote :

I had thought about trying to use LoggerFileObject in swift-init, but of course the exec'd processes had their own agenda for sys.stdout. I hadn't thought about modifying the daemon.run and run_wsgi code to be more aggressive about closing stdio (althought maybe they should flush?)...

Hrmm... doing a fork and exec with out closing stdout... leaving child processes open to spewing errors out and about your console after swift-init has terminated...

clayg@swift:~$ swift-init object start
WARNING: Unable to increase file descriptor limit. Running as non-root?
Starting object-server...(1269 - /etc/swift/object-server/1.conf)
Starting object-server...(1270 - /etc/swift/object-server/2.conf)
Starting object-server...(1271 - /etc/swift/object-server/3.conf)
Starting object-server...(1272 - /etc/swift/object-server/4.conf)
clayg@swift:~$
clayg@swift:~$
clayg@swift:~$ Error trying to load config /etc/swift/object-server/1.conf: File contains no section headers.
file: /etc/swift/object-server/1.conf, line: 1
'DEFAULT]\n'

clayg@swift:~$

You may be right that's it's better than getting blackholed, but I hadn't thought of that. Or maybe I'm misinterpreting your comment?

Another idea I had was to setup a default root logger if there's a problem with the conf and log errors on that instead of printing?

Another error that seemed to get lost was in validate_configuration - it was calling sys.exit("error message") after stderr redirection, but it didn't show up in syslog? Changing to print "error message"; sys.exit(1) seems to work just fine. I'm not really sure why it didn't get logged before...

Revision history for this message
Mike Barton (redbo) wrote :

Yeah, that's kind of ugly.

I bet swift-init could block reading the new process's stdout. Then when it's closed either because the child exits or dups over its stdout fileno, the read should return.

clayg (clay-gerrard)
Changed in swift:
status: New → In Progress
clayg (clay-gerrard)
Changed in swift:
milestone: none → 1.3.0
clayg (clay-gerrard)
Changed in swift:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in swift:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.