Charm needed: apache HBase

Bug #1487050 reported by amir sanjar on 2015-08-20
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juju Charms Collection
Undecided
Cory Johns

Bug Description

HBase is an open source, non-relational, distributed database modeled after Google's
BigTable and written in Java. It is developed as part of Apache Software Foundation's
Apache Hadoop project and runs on top of HDFS (Hadoop Distributed Filesystem),
providing BigTable-like capabilities for Hadoop.

* **Features**

- Linear and modular scalability.
- Strictly consistent reads and writes.
- Automatic and configurable sharding of tables
- Automatic failover support between RegionServers.
- Convenient base classes for backing Hadoop MapReduce jobs with Apache HBase tables.
- Easy to use Java API for client access.
- Block cache and Bloom Filters for real-time queries.
- Query predicate push down via server side Filters
- Thrift gateway and a REST-ful Web service that supports XML, Protobuf, and
binary data encoding options
- Extensible jruby-based (JIRB) shell
- Support for exporting metrics via the Hadoop metrics subsystem to files or Ganglia; or via JMX

Cory Johns (johnsca) wrote :

The status reporting on this could be improved significantly. When deployed with no relations, it indicates that it needs a relation to the plugin, but not Zookeeper nor the region server. Adding the relation to the plugin keeps it blocked, but the message still indicates that it's waiting for the plugin, instead of Zookeeper and the region server. Adding the relation to Zookeeper makes it indicate it's ready, but it can still be missing the regionserver relation and thus hbase will not be started.

I'm still of the opinion that having separate charms for separate roles makes things easier to understand, but I understand that there is a lot of overlap, even more-so in this charm than our others, so it's not a big deal here. I think composer and layers will make this easier in the future.

Once all of the relations were in place, it seemed to work as expected (smoke test worked). The smoke test should have a corresponding action, though.

Matt Bruzek (mbruzek) wrote :
Download full text (6.0 KiB)

Hello Amir! Thank you for the submission of apache hbase charm. I appreciate the amount of work that was done here to charm up this Apache project. I took a look at this charm and here is what I found.

# icon.svg
This icon does not look very nice. It is a white graphic on grey. The icon is on the same layer as the background so I was not able to fix this one, but I did create a new icon with a white background and put the hbase icon on the pictogram layer.

# README.md
The file converts properly to markdown and looks great! The quickstart command is invalid, and I am unable to locate the “apache-hadoop-hbase” bundle on jujucharms.com at all. The hyperlink to apache-hadoop-hbase url is also invalid giving me a 404 error.

# tests/
The tests appear to be a good representation of this charm. I ran them with bundletester and they passed! Congratulations on passing amulet tests! That helps verify the quality of the charm with a real deployment.

# master-relation-broken

While tearing down the test I encountered a hook error in the relations. Here is the “juju status”

      hbase-master/0:
        workload-status:
          current: error
          message: 'hook failed: "master-relation-broken" for hbase-regionserver:regionserver'
          since: 02 Oct 2015 11:31:52-05:00

The relevant section from the juju log:

2015-10-02 16:31:51 DEBUG juju.worker.uniter modes.go:31 [AGENT-STATUS] executing: running zookeeper-relation-departed hook
2015-10-02 16:31:51 INFO juju.worker.uniter.context runner.go:158 skipped "zookeeper-relation-departed" hook (not implemented)
2015-10-02 16:31:51 INFO juju.worker.uniter.context context.go:530 handling reboot
2015-10-02 16:31:51 INFO juju.worker.uniter.operation runhook.go:116 skipped "zookeeper-relation-departed" hook (missing)
2015-10-02 16:31:51 INFO juju.worker.uniter.operation executor.go:108 committing operation "run relation-departed (8; zookeeper/1) hook"
2015-10-02 16:31:51 INFO juju.worker.uniter.operation executor.go:68 running operation run relation-broken (10) hook
2015-10-02 16:31:51 INFO juju.worker.uniter.operation executor.go:108 preparing operation "run relation-broken (10) hook"
2015-10-02 16:31:51 INFO juju.worker.uniter.operation executor.go:108 executing operation "run relation-broken (10) hook"
2015-10-02 16:31:51 DEBUG juju.worker.uniter modes.go:31 [AGENT-STATUS] executing: running master-relation-broken hook
2015-10-02 16:31:52 INFO master-relation-broken Traceback (most recent call last):
2015-10-02 16:31:52 INFO master-relation-broken File "/var/lib/juju/agents/unit-hbase-master-0/charm/hooks/master-relation-broken", line 17, in <module>
2015-10-02 16:31:52 INFO master-relation-broken utils.run_as("ubuntu", spark_home+'/'+'stop-all.sh')
2015-10-02 16:31:52 INFO master-relation-broken TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
2015-10-02 16:31:52 INFO juju.worker.uniter.context context.go:530 handling reboot
2015-10-02 16:31:52 ERROR juju.worker.uniter.operation runhook.go:107 hook "master-relation-broken" failed: exit status 1

It looks like the value of spark_home is being used without checking if it is empty or None. I searched for a set of...

Read more...

Changed in charms:
status: New → In Progress
assignee: nobody → Cory Johns (johnsca)
Matt Bruzek (mbruzek) wrote :

It was brought to my attention that <email address hidden> has been replaced with <email address hidden> so please ensure the README.md is updated with the correct list.

Cory Johns (johnsca) on 2019-02-28
Changed in charms:
status: In Progress → Won't Fix
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers