Post Sample using correct pipeline when polling and noti. agent use different pipelines

[Rohit Jaiswal]

This affects juno and kilo.

When polling and notification agents use their own pipeline configurations (to ensure that polling and notification targets are mutually exclusive), it should be possible to use the POST sample API to post a sample for a meter/counter that is allowed by either polling or notification pipeline.

However, only one pipeline_cfg_file can be configured, which means only one pipeline can be used to filter the samples posted from API.

It should be possible to specify multiple pipeline configurations.

[Rohit Jaiswal (rohit-jaiswal-3)]

Why is this incomplete?

[gordon chung (chungg)]

i'm not sure why it's marked incomplete but just curious, in juno/kilo, the polling, notification and api services all have their own pipeline. they just happen to share the same one usually because the same ceilometer.conf file is used and thus the same pipeline.yaml is used... do you want to have multiple pipelines in across services or within a service?

[Rohit Jaiswal (rohit-jaiswal-3)]

So that the api service can use both notification and polling pipelines, it would be great to optionally be able to specify multiple pipelines for a service.

[gordon chung (chungg)]

so a posted sample would be handled by both polling and notification pipelines? what if there's duplicate/common pipelines? it seems like it requires a lot of validation to ensure you don't have random side effects.

[Rohit Jaiswal (rohit-jaiswal-3)]

Yeah, the alternative is to depend on admin to merge the pipes(source:sink combinations) from the polling and notification pipelines. Merge as in include the unique pipes and leave out duplicates so that the API uses all pipes to validate and filter in valid meters/samples. Or the validation can be done in pipeline.py:PipelineManager that uses the same logic.

[Rohit Jaiswal (rohit-jaiswal-3)]

I mean the only need is support to specify multiple sources i.e supported meter names, if polling and notifications support disjoint set of meters, the POST sample api should be able to use both sets.

[gordon chung (chungg)]

we only have one pipeline via notification agent. seems best to keep it simple