sub.sub_req always fails with "Reason 2437: FAILED: MQRC_NO_RETAINED_MSG" even if there are retained publications for the topic.

[Hannes Wagener]

Hi Dariusz,

As discussed in our previous emails...

I must be missing something on this one. I desperately need a second pair of eyes.

The new sub.sub_req method (which calls MQSUBRQ) always fails with: "Reason 2437: FAILED: MQRC_NO_RETAINED_MSG". I have changed a MQ example "c" program to call MQSUBRQ and that program returns the expected results.

At this stage I am not so sure anymore. I have been fishing around this one for a while. I have confirmed that the options passed in the "c" program are the same in the python one. I'm thinking it has something to do with identifying the topic - but the topic string looks fine. Other things like the options on both the SD and the SRO looks fine and compare between the c program and python.

The code can be found here: lp:~johannes-wagener/pymqi/retain-pub-issue

Please note: the code in the above branch contains debugging statements, experimentation, other tests and lots of output to stdout. I would suggest you only use "test*" in the "src" directory along with the current pub-sub branch revision(lp:~pymqi-dev/pymqi/pub-sub).

This is how to reproduce:
1.) run test_pub_retain.py. This will publish a message with the CMQC.MQPMO_RETAIN option.
2.) run test_sub_from_sub_not_managed_durable_retained_publications.py (this is the one thefails with 2437: please note that the real options used in this script does not match it's name but rather what is in the c program.). Even though the options in the c program and the python script matches the python version never finds any retained publications for any topic.
3.) run the test_retained_sub "c" program. You will notice that he returns the retained publication every time no matter how many times you run it.

I have not had an opportunity to run this on a different system(even though I doubt it's environment related) - but I will make some time to build it on windows tomorrow.

Your help is greatly appreciated.

Cheers,

H.

[Dariusz Suchojad (dsuch)]

Err, Hannes, are you sure you've commited test_pub_retain.py and test_sub_from_sub_not_managed_durable_retained_publications.py to the repo? I can see retain-pub-issue/src/test_retained_sub.c but not the other files you mentioned?

Not sure if that's the case or am I simply doing something very stupid :-) but if it's the former then maybe setting these aliases can help in the future?

bzr alias commit="commit --strict"
bzr alias push="push --strict"

[Dariusz Suchojad (dsuch)]

Ah and by the way, what MQ version are you using? There are APARs PM20443 and PM25601 regarding MQPMO_RETAIN and although they don't seem to be that much related one cannot always be sure, the pub/sub thing is quite new so who knows what's still lurking there.

http://www-01.ibm.com/support/docview.wss?uid=swg1PM20443
http://www-01.ibm.com/support/docview.wss?uid=swg1PM25601

[Dariusz Suchojad (dsuch)]

OK, I still don't know how to properly do it on Python side but I've created two simple C programs for pub/sub with MQSUBRQ, they're in the attachments (retained-put.c & retained-get.c).

Before running them you need to define a topic like that one DEFINE TOPIC(TOPIC.1) TOPICSTR('/topic/1') and compile the programs

$ gcc -Wall -I/opt/mqm/inc -c retained-put.c && gcc ./retained-put.o -L/opt/mqm/lib64 -lmqic_r -o ./retained-put
$ gcc -Wall -I/opt/mqm/inc -c retained-get.c && gcc ./retained-get.o -L/opt/mqm/lib64 -lmqic_r -o ./retained-get

You can then run them like below

$ ./retained-put
Channel: SVRCONN.1
Connection name: 192.168.1.139(1434).

MQCONNX ended with reason code 0
MQOPEN ended with reason code 0
MQPUT ended with reason code 0
MQCLOSE ended with reason code 0
MQDISC ended with reason code 0
$

$ ./retained-get
Channel: SVRCONN.1
Connection name: 192.168.1.139(1434).

MQCONNX ended with reason code 0
MQSUB ended with reason code 0
MQSUBRQ ended with reason code 0
sro.NumPubs: 1
MQGET ended with reason code 0
message <My payload>
MQDISC ended with reason code 0
$

In other words, if we have it working in C then making it work in Python should be doable as well. There are two interesting points:

- retained-put.c uses the name of a topic whereas retained-get.c has to use the topic's string not its name
- I couldn't get it working until I made it two separate programs, it was just one initially but I couldn't issue MQGET without it raising MQRC 2033. It magically disappeared when I made it two programs instead.

Well, it's not much but at least it's a confirmation that the environment works OK. Will try to tackle the Python side now.

[Dariusz Suchojad (dsuch)]

OK, here's some more food for mind. Most apparently, MQRC 2437 MQRC_NO_RETAINED_MSG is returned not only when there are no messages for a legitimate topic string but also when the topic string doesn't exist at all (shame IBM for using the same code for both cases).

I did an interesting experiment - right before calling MQSUBRQ in pymqe_MQSUBRQ, I added /yet another/ hardcoded subscription for the topic string, that is I discarded the previous subscription, made another one and all of a sudden the call to MQSUBRQ would start working.

In that case I think there's something wrong with how the subscription object/handle/descriptor/who knows what is being passed from Python to C and back. It may be a long shot but I'd say 'sd.ObjectString.VSPtr' and 'sd.ObjectString.VSLength' are somehow being dropped along the way and because MQ returns the same code regardless of the reason - whether it's actually no messages or a non-existing topic string (like when sd.ObjectString is not set) - well, I think that may be the reason. That's it for now, will let you know if I come up with anything.

[Hannes Wagener (johannes-wagener)]

Thanks! Will check those APARS. I'm on 7.0.1.3. Will also make sure I checked in all the code as soon as I am home.

>I did an interesting experiment - right before calling MQSUBRQ in pymqe_MQSUBRQ, I added /yet another/ hardcoded >subscription for the topic string, that is I discarded the previous subscription, made another one and all of a sudden the call to >MQSUBRQ would start working.

What? That's strange. But why only in python then. I do exactly in "c" what I do in python but get different results. I will once again check and double check the VSPtr fields at all levels and I will check the subscription handle.

The thing about using topicString, topicName and subName heavily depends on the subscription options you sepcify. If your subscription is unmanaged you always have to supply a subName. The use of topicName and/or topicString is as documented in the MQ manual.

Thanks for your help. I'm going to look at this with new eyes after your experiment.

[Dariusz Suchojad (dsuch)]

OK Hannes, I'd appreciate it if you could have a look at it because I know I'll be busy this week. Will join the efforts around Friday though!

[Dariusz Suchojad (dsuch)]

Hi, I'm back on track with resolving this issue. Have you managed to come up with anything? It's OK if you haven't, I'll be digging into it myself now and will keep you updated!

[Dariusz Suchojad (dsuch)]

Hm, about we don't deal with it for PyMQI 1.2? Really, I can see we're both busy with other projects and the more time we spend on this issue, the longer people won't be allowed what other cool features there are already available in 1.2. So if you don't protest, I'll start on merging the pub/sub stuff up to trunk but it won't include MQSUBRQ for now.

[Hannes Wagener (johannes-wagener)]

Sorry was away. This is really turning out to be a hard one to solve.

Congrats on 1.2. I see you did a sterling job at updating the documentation!

Thanks for that. Will let you know when I have finally fixed this issue.

[Dariusz Suchojad (dsuch)]

Setting it to Invalid for now - there has been no activity for quite some time.