New Charm: IBM MQ

[vsr]

Hi!

Here is a new charm for IBM MQ for review.

To test this charm, a license to IBM MQ is required.
You need to get an IBM MQ license for at least MQ 8.0.0.0 (for x86_64) or MQ 8.0.0.2 (for Power LE/x86_64)

Repo: https://code.launchpad.net/~ibmcharmers/charms/trusty/ibm-mq/trunk

[amir sanjar (asanjar)]

Thanks for all of your work putting IBM-MQ charm together for review. We require a license file/key for testing purpose only, what is the process to obtain such license.

[Antonio Rosales (arosales)]

Hello, and thanks again for your contribution to the Juju community.

In reviewing your charm I had the following comments:

Clarify license acceptance:
 - In the README.md suggest under the "Usage" section, as done in the "Configuration" section, to specifically tell the user that license file can only be read after downloading and extracting the IBM MQ contents to files/archive. I was initially under the impression that I could read the terms and conditions in files/archive before downloading the IBM MQ contents.
 - Given that the README.md tells me the license can be read only after downloading the IBM MQ software and extracting into files/archive the config parameter, "accept-ibm-mq-license" is a little confusing as it states the IBM-MQ-license.txt in the root directory must be read and accepted. This file is also of type "data" and not text.

If a license needs to be set in order for the software to work suggest to provide the IBM MQ binary in the charm with the terms and conditions that must be set to true per a specified license/terms-of-conditions text file as already done with the accept-ibm-mq-license config option. A new config option can then be made to set the license key.

Specify the exact location in the README.md where the IBM MQ license can be obtained.

-thanks,
Antonio

[Antonio Rosales (arosales)]

As an fyi be sure to move the status of this bug back to "Fix Committed" when you have addressed the above comments. Moving the bug back to "Fix Committed" will put this bug back into the review queue for a subsequent review.

-thanks,
Antonio

[vsr (vsasi)]

Amir, hope this link will help: https://www.ibm.com/developerworks/community/blogs/messaging/entry/develop_on_websphere_mq_advanced_at_no_charge?lang=en

[Antonio Rosales (arosales)]

@vsr,

Thanks for the info. This review is marked Incomplete per comment 2. Please address comment 2, and when complete move the status of this bug back into "Fix Committed." As comment 2 is still outstanding I will move the status of this review to "Incomplete."

-thanks,
Antonio

[vsr (vsasi)]

@Antonio,

Did you have chance to check the updated README and config.yaml?

I had checked in the README with updates before moving the status to Fix Committed. This includes the following statement:

To install the downloaded binaries you must agree to the IBM license. The license file(s) can be found under the files/archives directory on extraction of IBM MQ software. The license can also be viewed by running the following command after extracting IBM MQ to the files/archives directory:
      ./mqlicense.sh -text_only

Also had updated the config.yaml.

It is not feasible to ship the binaries themselves as they are licensed.

Is there anything else that was needed?

It is not feasible to ship the binaries themselves.

[Antonio Rosales (arosales)]

@Vsr,

Thank you for the updates to the README to clarify the license acceptance process.

To confirm on the binaries:
> It is not feasible to ship the binaries themselves as they are licensed.

Understood the binaries are licensed, however can you help me understand how they are licensed so we can find the best way forward for users. Does a user have to set a license key on deploy of the MQ software, or before downloading the software? Also said another way, does the MQ software work without a user setting a license key?

I also have the following comments that need addressed per my initial review:
>> Specify the exact location in the README.md where the IBM MQ license can be obtained.

Given there are a lot of questions on how to obtain the MQ software from the review potential users may also run into this. Suggest to detail how the users gets the MQ binary (what URL to visit, what web workflow to follow, what points to take note of during the web workflow), and then once the user downloads the binary detail how the user places the binary into the charm so it can be deployed locally. Also note in the caveats that the charm must be deployed locally if we can not ship the binaries in the charm and license set. Also suggest how the user can download the "BM MQ 90 day no-charge trial."

Also suggest in the "Overview" to tell what the charm does and a link to more information, such as:
  http://www-03.ibm.com/software/products/en/ibm-mq
as the "IBM MQ Information" section links to the license and not the readme. Given that suggest in the "IBM MQ Information" section have two links; (1) General info and (2) license.

On a side note if you want your system configuration parameters to read as code (which does read nicer) you'll need to follow the indention MarkDown rules at:
  https://guides.github.com/features/mastering-markdown/

For now I'll move this bug into "Incomplete" while the above items are addressed. Please update the status to "Fix Committed" when you are ready for another review. Thanks again for your work on this charm to ensure a great user experience.

We can work separately on how to test the charm for the review and in CI going forward.

-thanks
Antonio

[Antonio Rosales (arosales)]

@Vsr,

Also on the icon please head the star badge icon.

-thanks,
Antonio

[vsr (vsasi)]

@Antonio,

I wanted what is meant by : 'head the star badge icon'

Thanks,
Sasi

[José Antonio Rey (jose)]

@vsr He probably meant 'hide' and not 'head'. Bear in mind, the star cannot be shown on the icon, it needs to be just the icon design with no star on top.

[vsr (vsasi)]

@Antonio,

  I have update the README with links to documents on how to download the software and links to the documentation of the software. I have not elaborated on the actual steps for the download since those could change over time.

The license acceptance can be set after the download of the software (there is no separate key as such for accepting the license). The mechanism is elaborated in the README.

The icon file is updated with the star removed.

Thanks,
Sasi (vsr)

[Antonio Rosales (arosales)]

@Vsr,

Thanks for the updates.

>I have update the README with links to documents on how to download the software and links to the documentation of the >software.

Thanks for the updates to the readme. A lot more helpful with folks getting started with IBM MQ. There were still a few small fixes which I made which you can review at your convenience at:
https://code.launchpad.net/~a.rosales/charms/trusty/ibm-mq/update-readme/+merge/256411

>I have not elaborated on the actual steps for the download since those could change over time.

Understood, and your footnote in section (2) seems very appropriate to address this.

>The license acceptance can be set after the download of the software (there is no separate key as such for accepting the license). >The mechanism is elaborated in the README.

Given the software is non-functional without setting keys would be it be worth exploring the charm deliving the MQ binary and setting the license key as a config option at deploy time?

>The icon file is updated with the star removed.

We are coordinating with IBM to obtain binaries and with the 90 day trail information we are also leveraging that in testing the binary. We will post our results to the bug once we obtain binaries.

-thanks,
Antonio

Thank you.

[Matt Bruzek (mbruzek)]

ibm-mq review

# Review
Thank you for authoring the IBM MQ product into a charm! I can tell that a lot of effort went into this charm. I really appreciate that our tools pass such as `charm proof`. I have made some suggestions in my review to make this charm work even better.

# charm proof
Running charm proof returns no errors or warnings, that is awesome thank you!

# icon.svg
We generally try to avoid letters in the icon, but this icon is OK if the letters “MQ” is the only logo for the IBM MQ product.

I opened the icon.svg in inkscape and it does not appear to have any layers. I would highly suggest moving the current icon to a back up then generating another template.
'''
mv icon.svg first_draft.svg
charm add icon
inkscape icon.svg
(edit the PICTOGRAM layer and the background if required)
save the image
```
Then follow the icon documentation:
https://jujucharms.com/docs/1.20/authors-charm-icon
Put the logo in the “PICTOGRAM” layer for best results. It looks like others had commented on the icon and if you are having trouble using inkscape or the other tools please reach out to me, I would be happy to help you get this right.

# copyright
The copyright for the charm code looks to be Apache 2.0, that works for us!

One thing that is done on other IBM charms is including a license file for the IBM MQ license. This would provide a way for users to read and understand the MQ license that they must accept in order for the charm to deploy and install. Perhaps copy the server/readmes/readme_en_US file into the charm directory, or include all languages in a license directory.

# metadata.yaml
The metadata looks fine. The maintainer field supports a name before the email address. It might help to understand the maintainer is not a person but a group. Juju Support team <email address hidden> or something like that.

# hooks/install
That is a strange approach to use RPM to install the package, I have never seen that in a charm before! Most charm code uncompresses a tar image, but if this works using rpm is fine (just unconventional).

# hooks/config-changed
The bash code is clean and easy to understand! Thank you for the work here!

# README.md
The readme converts to HTML and looks to be correct Markdown syntax.

The instructions on how download the software are for a Power only and do not match the name that I found for x86_64. May I suggest generalizing the name to a variable or giving an example for Power and x86_64 (the supported platforms).

The steps in the readme are wrong, and should be addressed. The instructions result in `files/archives/server/` directory. Notice “archives” is plural (s) and the additional `server` directory. The config-changed code checks in the `$CHARM_DIR/files/archives/...` directory and does not see the files inside the server directory.

The readme does not cover what charms or services can be related to this charm. I was curious about what charms relate to this and how this charm can be used.

The readme will be converted to HTML and be the first page users see in the charm store. All of the hyperlinks in the readme the text “here”. When writing documentation it is more clear if you use desc...

[Matt Bruzek (mbruzek)]

@Vsr,

How does one verify that the IBM MQ is properly installed and running? Ideally the tests should do this, but I want to know how to tell if MQ is running corretly. I fixed all the charm errors by hand and the config-changed hook completes successfully, but I don't see a system process called MQ, or anything listening on port 1414.

Thanks!

[Matt Bruzek (mbruzek)]

@Vsr,

Can you also add the link to the 90 day trial to the readme? I do not see it anywhere in there.

https://www.ibm.com/developerworks/community/blogs/messaging/entry/develop_on_websphere_mq_advanced_at_no_charge?lang=en

[vsr (vsasi)]

Thanks for the review and comments.

@Antonio,

The link https://code.launchpad.net/~a.rosales/charms/trusty/ibm-mq/update-readme/+merge/256411 is giving Page not found error.

The license key setting is provided as a config option. Default is false as the user necessarily needs to accept it.

@Matt

Icon: Need to work on it and might need your help. Did not want to hold up the review for this.

License to be part of charm: Charm to be independent of the software license allowing users the flexibility of using a different version of MQ - which could perhaps have a different license.

Related charms: No related charms yet, but there are thoughts to develop those. Currently, once MQ is installed and then configured, it can be used by MQ client software.

Binary : Awaiting response about whether it is feasible to ship it with the charm.

[vsr (vsasi)]

@Matt

Updated the icon

[amir sanjar (asanjar)]

thanks vsasi, icon looks acceptable.

[Matt Bruzek (mbruzek)]

Thanks for the update on the IBM MQ charm. I reviewed the latest version and here is what I found.

I was able to download the developer version of IBM MQ linked in the README, and the charm deployed with this binary. Consider editing the README file to include download links near the Usage section. The README should also contain a “## Known Limitations and Issues” section to address the limitations on the “local” provider (that it can not use the “sysctl” commands). I deployed this charm on the local provider, and the sysctl commands in the test directory will not pass tests on the “local” (LXC) provider (but you could use the same logic in the charm to prevent calling the sysctl commands in the tests).

The mixture of spaces and tabs in the hooks is inconsistent and does not line up on some editors. Although bash allows a mixture of spaces and tabs it is best practice to use one or the other. Please edit all the hooks and use either spaces or tabs so the code lines up correctly.

Should the MQ service be running as a result of a properly deployed charm? When deploying and accepting the license I would expect to see the MQ service set up and running. This does not seem to be the case here. Suggest creating and starting a queue after installing the software. I would have expected to see IBM MQ running after the charm is started. See comment #14.

Thank you for writing more tests. Looking at the code you only create a queue and start the queue. A good test for a queue is to write a message and read a message from the queue to make sure it works. The current tests failed to run because they are not executable! After adding the executable permission to 00-setup, and 10-bundles-test.py the test failed with this message: “TabError: inconsistent use of tabs and spaces in indentation”. Python requires that you use 4 spaces for code blocks. Please run flake8 or pep8 against the python test file and fix any issues those tools report. If you are interested in how we test, we use bundletester ( https://github.com/juju-solutions/bundletester ) which you can install on a system, and I use the following options “bundletester -F -e amazon -l DEBUG -v”. Please make sure the tests run and pass before the next review. The commands run in the tests "sysctl" will fail on "local" environments so use the same logic in the config-changed hook to prevent calling the commands if they can not be called.

Can the IBM MQ software be downloaded by the IBM Installation Manager? A colleague of mine suggested using Installation Manager sofware in the charm could eliminate the user having to download the IBM MQ software ahead of time. Please look into this option as it would make the charm so much easier for users to install.

Thanks again for the contribution of time and effort. I am going to put this bug back into “In Progress” while you work on the issues found in this review. If you want to discuss the review please join us in #juju on freenode, my user id is “mbruzek”. If you have any other questions or concerns please email <email address hidden> or email me directly.

[vsr (vsasi)]

@Matt
Thanks for the review.

Added the Limitation section for local provider
Corrected the formatting issues with bash script

MQ service will run once a queue manager is started.

Since creating queues require user input/reading from a file, added a verification step in the charm to do the same.

Changed the test scripts to execute mode and ran against pep8 with no messages. Changed FAIL to SKIP for system configuration setting failures.

MQ does not use Installation Manager and so would not be available to download using the same. Trail version of MQ should be available at a location set up by Antonio.

[vsr (vsasi)]

MQ trial version available @: https://bazaar.launchpad.net/~ibm-charmers/ibm-juju-sw/ibm-mq-8.0/files

[Adam Israel (aisrael)]

Hi Vsasi,

Could you give us an md5sum of the trial version? We're seeing the following gzip errors while trying to extract the MQ trial version at the above link.

server/repackage/MQSeriesSamples.spec
server/MQSeriesExplorer-8.0.0-2.x86_64.rpm

gzip: stdin: invalid compressed data--format violated
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now

[Cory Johns (johnsca)]

Vsasi,

I got the gzip error mentioned above on the development download mentioned in the README, as well.

Additionally, I noticed some formatting issues in the README, which I took the liberty of cleaning up for you in lp:~johnsca/charms/trusty/ibm-mq/review (https://code.launchpad.net/~johnsca/charms/trusty/ibm-mq/review).

In that branch, I also made some suggested improvements to the test case, changing the amulet.raise_status() calls to asserts, since raise_status() is not handled properly by the test runners such as bundletester, which is our standard test runner for review and automated testing, and always cause the test to report as failure. Additionally, the tests/bundles.yaml file was marked as executable, causing bundletester to attempt to run it.

Finally, in the test, I noticed that it was simply repeating some of the configuration steps from the charm, instead of checking to ensure that the charm performed the steps correctly. For example, instead of having the test attempt to add the ubuntu user to the mqm group, I changed it to use the "groups" command to verify that it was added successfully to the group by the charm. There were a couple of cases at the bottom of the test, for starting the charm's services, that I wasn't sure how to convert to checks instead of attempts to re-perform the steps from the charm.

[vsr (vsasi)]

Trial Version -
$ md5sum WSMQ_8.0_IFR_TRIAL_LNX_ON_X86_64_.tar.gz
682bd99df75a24e78d5a3776bf6acdcd WSMQ_8.0_IFR_TRIAL_LNX_ON_X86_64_.tar.gz

File size(bytes): 578899344 WSMQ_8.0_IFR_TRIAL_LNX_ON_X86_64_.tar.gz

Development Version -
$ md5sum mqadv_dev80_linux_x86-64.tar.gz
6cdb0c142b6ca7de3b92d18db1437da7 mqadv_dev80_linux_x86-64.tar.gz

File size(bytes): 578360499 mqadv_dev80_linux_x86-64.tar.gz

[vsr (vsasi)]

Thanks for the review and updated files. I have merged those into the trunk.

Johnsca, about the services at the end - please note the charm deletes the queue manager. So it is tested again in the script.

[vsr (vsasi)]

MQ package is uploaded to canonical.brickftp.com

[Matt Bruzek (mbruzek)]

Sasi

Thanks for the latest submission of this ibm-mq charm. I took a look at the charm and the tests passed so I promulgated the charm to the recommended name space. Congratulations!

There has been several iterations of reviews so thank you for sticking with the review process. We have created a higher quality charm because of the reviews by several people. I appreciate the time and effort put forth to get this charm in the recommended namespace.

The charm now has launchpad repository in the charmers namespace:

https://code.launchpad.net/~charmers/charms/trusty/ibm-mq/trunk

From now on to update the charm you start with this branch. Also note that the bugs for this charm will be listed: https://bugs.launchpad.net/charms/+source/ibm-mq

It will take about 30 minutes for the charm to show up in the charm store, but there will be the shorter namespace for this charm.

Congratulations again! Thanks again for all the your efforts!

[Anita Nayak (anitanayak)]

Hi Matt,

Updated ibm-mq charm to get reflect the config changes for soft and hard nofile.
Also updated README.md file to update the system parameters manually.

Please review the changes.

Details of code check-in to launchpad and charm store:

Deployable Charm Repo : https://code.launchpad.net/~ibmcharmers/charms/trusty/ibm-mq/trunk

IBM-MQ charm uses interface-ibm-mq. Here is the source code link:
https://code.launchpad.net/~ibmcharmers/charms/trusty/interface-ibm-mq/trunk

The charm can be deployed from
https://jujucharms.com/u/ibmcharmers/ibm-mq/trusty/18

Thanks

[Anita Nayak (anitanayak)]

Hi Matt,

Updated ibm-mq charm to get reflect the config changes for soft and hard nofile.
Also updated README.md file to update the system parameters manually.

Please review the changes.

Details of code check-in to launchpad and charm store:

Deployable Charm Repo : https://code.launchpad.net/~ibmcharmers/charms/trusty/ibm-mq/trunk

IBM-MQ charm uses interface-ibm-mq. Here is the source code link:
https://code.launchpad.net/~ibmcharmers/charms/trusty/interface-ibm-mq/trunk

The charm can be deployed from
https://jujucharms.com/u/ibmcharmers/ibm-mq/trusty/19

Thanks,
Anita

[Anita Nayak (anitanayak)]

Fixed the lint errors.

Please review final changes.

Details of code check-in to launchpad and charm store:

Deployable Charm Repo : https://code.launchpad.net/~ibmcharmers/charms/trusty/ibm-mq/trunk

IBM-MQ charm uses interface-ibm-mq. Here is the source code link:
https://code.launchpad.net/~ibmcharmers/interface-ibm-mq/trunk

The charm can be deployed from
https://jujucharms.com/u/ibmcharmers/ibm-mq/trusty/20

Thanks,
Anita