Nova servers should have separate configuration files

[Jay Pipes]

Nova servers should have separate configuration files and use common bind_host/bind_port/SSL options. Makes things easier to configure and removes the need for separate flags like ec2_listen, osapi_volume_listen, metadata_listen and osapi_listen.

In addition to simplifying configuration, we can get the following benefits:

 * Would be able to use a standard WSGI Server class
 * Would be able to take advantage of the SSL and socket-wrapping work done in Glance and Swift
 * Would be able to take advantage of the multi-processed servers in Glance and Swift

Essentially, each server/service config would have this:

[DEFAULT]
# Address to bind the server
bind_host = 0.0.0.0

# Port the bind the server to
bind_port = 9292

# Backlog requests when creating socket
backlog = 4096

# Number of worker processes to start.
# On machines with more than one CPU increasing this value
# may improve performance (especially if using SSL with
# compression turned on). It is typically recommended to set
# this value to the number of CPUs present on your machine.
workers = 0

as well as (eventually) all the SSL configuration options like cert_file, key_file, etc

No need for duplicate cfg options for all the different services. Instead, all the services define a common bind_host / bind_port and the WSGI socket handling code can be standardized to what is in Glance (and Swift without the openstack-common cfg module...):

bind_opts = [
    cfg.StrOpt('bind_host', default='0.0.0.0'),
    cfg.IntOpt('bind_port'),
]

socket_opts = [
    cfg.IntOpt('backlog', default=4096),
    cfg.StrOpt('cert_file'),
    cfg.StrOpt('key_file'),
]

workers_opt = cfg.IntOpt('workers', default=0)

def get_bind_addr(conf, default_port=None):
    """Return the host and port to bind to."""
    conf.register_opts(bind_opts)
    return (conf.bind_host, conf.bind_port or default_port)

def get_socket(conf, default_port):
    """
    Bind socket to bind ip:port in conf

    note: Mostly comes from Swift with a few small changes...

    :param conf: a cfg.ConfigOpts object
    :param default_port: port to bind to if none is specified in conf

    :returns : a socket object as returned from socket.listen or
               ssl.wrap_socket if conf specifies cert_file
    """
    bind_addr = get_bind_addr(conf, default_port)

    # TODO(jaypipes): eventlet's greened socket module does not actually
    # support IPv6 in getaddrinfo(). We need to get around this in the
    # future or monitor upstream for a fix
    address_family = [addr[0] for addr in socket.getaddrinfo(bind_addr[0],
            bind_addr[1], socket.AF_UNSPEC, socket.SOCK_STREAM)
            if addr[0] in (socket.AF_INET, socket.AF_INET6)][0]

    conf.register_opts(socket_opts)

    cert_file = conf.cert_file
    key_file = conf.key_file
    use_ssl = cert_file or key_file
    if use_ssl and (not cert_file or not key_file):
        raise RuntimeError(_("When running server in SSL mode, you must "
                             "specify both a cert_file and key_file "
                             "option value in your configuration file"))

    sock = None
    retry_until = time.time() + 30
    while not sock and time.time() < retry_until:
        try:
            sock = eventlet.listen(bind_addr, backlog=conf.backlog,
                                   family=address_family)
            if use_ssl:
                sock = ssl.wrap_socket(sock, certfile=cert_file,
                                       keyfile=key_file)
        except socket.error, err:
            if err.args[0] != errno.EADDRINUSE:
                raise
            eventlet.sleep(0.1)
    if not sock:
        raise RuntimeError(_("Could not bind to %s:%s after trying for 30 "
                             "seconds") % bind_addr)
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    # in my experience, sockets can hang around forever without keepalive
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)

    # This option isn't available in the OS X version of eventlet
    if hasattr(socket, 'TCP_KEEPIDLE'):
        sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPIDLE, 600)

    return sock

[Davanum Srinivas (DIMS) (dims-v)]

Should probably start from the glance copy of this code - https://github.com/openstack/glance/blob/master/glance/common/wsgi.py

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/19452
Committed: http://github.com/openstack/oslo-incubator/commit/e50b68c0d28cb00fa627525f23bb0c0f614b9312
Submitter: Jenkins
Branch: master

commit e50b68c0d28cb00fa627525f23bb0c0f614b9312
Author: Davanum Srinivas <email address hidden>
Date: Fri Jan 11 00:10:56 2013 -0500

    Support for SSL in wsgi.Service

    Enable support for SSL as well using code from glance. We
    have some new options for configuring the SSL support.

    test_app starts wsgi.Service with a test app, then creates
    a url to make sure the http requests are actually served
    properly

    test_app_using_router adds wsgi.Router and Mapper() to the
    mix along with using the wsgify annotation for serving
    the http requests

    Fixes LP# 979488 (partial)
    Fixes LP# 869884 (partial)

    DocImpact

    Change-Id: Iae47b13b50e00c102c8c36f4a3e73b24fa4e6303

[Joe Gordon (jogo)]

Jay, do you still want to reorignize the config files? If so we should re-open this bug under oslo (oslo.config). Marking as opinion for nova, feel free to change it back.

[Jay Pipes (jaypipes)]

LOL, this was a long time ago. I still think it's a worthwhile effort, but extremely low priority.