Remove IPA Eventlet Dependency

Bug #2062954 reported by Adam McArthur
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
ironic-python-agent
Triaged
Medium
Unassigned

Bug Description

For more information on "Moving Away From Eventlet" in Openstack, see https://review.opendev.org/c/openstack/governance/+/902585 for a good summary.

Current State-of-World
=================================================================
IPA Depends on eventlet heavily through its use of oslo.service WSGI.

The hope originally was that IPA would be the easiest to migrate away from eventlet, by simply applying the already built WSGI application to something non-eventlet based (Gunicorn, Unicorn etc), and at the same time get oslo.service WSGI migrated as well.

This work can be tracked here: https://etherpad.opendev.org/p/posteventlet-networking.

**Currently it is assumed that this original plan is not possible.**

The "New" Problem
=================================================================
IPA was designed to spawn its own WSGI Servers as required. As well as this, the IPA WSGI Application that the WSGI Server Runs, also needs to be able to control IPA (i.e request comes to the API, the agent takes an action)

This creates an interesting problem, because the agent needs to be able to make thread-safe operations. The agent is currently past through to Application to enable this flow. https://github.com/openstack/ironic-python-agent/blob/master/ironic_python_agent/api/app.py#L90

**This means that just changing the server is not enough, the flow of data through the application also needs to change.**

High Level Plan
=================================================================

Based on the work of JayF and TheJulia - thank you :))

- Remove IPAHeartbeater dependency on agent (https://github.com/openstack/ironic-python-agent/blob/cdd0a834483d8c19324fc06d8a836ccf28b2424f/ironic_python_agent/agent.py#L76)
directly require, as arguments, the data it is currently reading from self.agent
    - move get_agent_advertise_address up a level and pass into Agent

- Create a class to hold most of the runner/instantiation code for IronicPythonAgent class
    - The goal: to ensure things needed both by WSGI application and IronicPythonAgent class to be gotten
      one and passed in explicitly, instead of being shared via agent class

- Where dynamic data needs to be shared
    - (Option 1) Develop database backend (sqlalchemy+sqlite?)
        - Ensure we never have two writers!!!
        - The API server side *does not have to be* WSGI when implementing this
    - (Option 2) Use IPC (JSON-RPC over a socket) to call from an independent, statelss API/wsgi
       application into a backend/conductor application
    - This would mimic the configuration from Ironic API -> Ironic Conductor

Tags: rfe
Revision history for this message
Jay Faulkner (jason-oldos) wrote :

I'd like to note when we get into action items: removing the heartbeater dependency on agent, and restructuring IPA startup are obvious wins -- how we do the third step is still up for debate a little bit, and this bug exists to document some of that planning.

Revision history for this message
Dmitry Tantsur (divius) wrote :

> (Option 1) Develop database backend (sqlalchemy+sqlite?)

OH GOD

> (Option 2) Use IPC (JSON-RPC over a socket)

PLEASE NO :)

Primitives like locks around shared memory or queues are boring but still work. IPA is essentially built around running a single task, having one API thread and one executor thread connected via a queue should actually be enough.

Changed in ironic-python-agent:
status: New → Triaged
importance: Undecided → Medium
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.