drizzle is using MySQL option files

[Giuseppe Maxia]

the drizzle server (drizzled) and launcher (drizzled_safe, provided that we decided to preserve it or replace with something else), and the drizzle clients (bin/drizzle*) are currently using MySQL option files.

/etc/my.cnf
/etc/mysql/my.cnf
$BASEDIR/etc/my.cnf
$HOME/.my.cnf

How to repeat:
===========
(1)
* create a $HOME/.my.cnf file containing the following
[mysqld]
fail-here

* launch drizzled

(2)
* create a $HOME/.my.cnf file containing the following
[client]
database=not_existent

* launch drizzled
* launch drizzle

Why this is serious
==============
options intended for MySQL are not necessarily intended for Drizzle. They must have a separate set of option files.

[Monty Taylor (mordred)]

This is on my TODO list...

What we've talked about so far is removing "my.cnf" as the thing that's configured and instead having a conf dir location (like /etc/drizzle) Within that location, various things can look for things like "drizzled.conf" "client.conf" etc.

For now, though, and for this bug, I'll just replace references to my.cnf with drizzle.cnf and /etc/mysql with /etc/drizzle...

[Giuseppe Maxia (giuseppe-maxia)]

On Tue, Jul 29, 2008 at 4:10 AM, Monty Taylor <email address hidden> wrote:
>
> For now, though, and for this bug, I'll just replace references to
> my.cnf with drizzle.cnf and /etc/mysql with /etc/drizzle...
>

If you do this change, please keep two option files in the user controlled space

* $BASEDIR/drizzle.cnf or $DATADIR/drizzle.cnf
* $HOME/.drizzle.cnf

access to /etc/drizzle... requires root privileges. We must preserve
the flexibility of dealing with drizzle in user space.

Giuseppe

[shinguz (oli-sennhauser)]

Hi all

All this different locations where a DB can be controlled I personally think is a nightmare (> 4 file locations +
command line).

My vote is for ONE file.

But if not ONE file, could we have at least a variable where is noted, which sources where considered and in which
order?

For example

SHOW GLOBAL VARIABLES LIKE 'configuration';
...
| configuration | /etc/my.cnf:/home/oli/.my.cnf:command-line |
...

Regads Oli

PS: Giuseppe: What do you need the $HOME/.my.cnf for? Why are you not just linking the /etc/my.cnf to
/home/mysql/data_123/my.cnf?
(The approach with /etc/my.cnf does only work with one instance per server anyway and then you have to have several
my.cnf...)

----Ursprüngliche Nachricht----
Von: <email address hidden>
Datum: 29.07.2008 04:31
An: <email address hidden>
Betreff: Re: [Bug 252780] Re: drizzle is using MySQL option files

On Tue, Jul 29, 2008 at 4:10 AM, Monty Taylor <email address hidden> wrote:
>
> For now, though, and for this bug, I'll just replace references to
> my.cnf with drizzle.cnf and /etc/mysql with /etc/drizzle...
>

If you do this change, please keep two option files in the user
controlled space

* $BASEDIR/drizzle.cnf or $DATADIR/drizzle.cnf
* $HOME/.drizzle.cnf

access to /etc/drizzle... requires root privileges. We must preserve
the flexibility of dealing with drizzle in user space.

Giuseppe

--
drizzle is using MySQL option files
https://bugs.launchpad.net/bugs/252780
You received this bug notification because you are a member of Drizzle-
developers, which is subscribed to Drizzle.

Status in A Lightweight SQL Database for Cloud and Web: Confirmed

Bug description:
the drizzle server (drizzled) and launcher (drizzled_safe, provided that we decided to preserve it or replace with
something else), and the drizzle clients (bin/drizzle*) are currently using MySQL option files.

/etc/my.cnf
/etc/mysql/my.cnf
$BASEDIR/etc/my.cnf
$HOME/.my.cnf

How to repeat:
===========
(1)
* create a $HOME/.my.cnf file containing the following
[mysqld]
fail-here

* launch drizzled

(2)
* create a $HOME/.my.cnf file containing the following
[client]
database=not_existent

* launch drizzled
* launch drizzle

Why this is serious
==============
options intended for MySQL are not necessarily intended for Drizzle. They must have a separate set of option files.

[Monty Taylor (mordred)]

I agree on 1 for the server... There should be one location - not a
search path of locations that all add together (or not).

However, I _do_ think that 2 locations for client program is good and
useful - one system wide and one per user (/etc/drizzle/client.conf &&
~/.drizzle.conf) ... this is pretty standard across the unix world...

But the many files in a path thing is terrible.

shinguz wrote:
> Hi all
>
> All this different locations where a DB can be controlled I personally think is a nightmare (> 4 file locations +
> command line).
>
> My vote is for ONE file.
>
> But if not ONE file, could we have at least a variable where is noted, which sources where considered and in which
> order?
>
> For example
>
> SHOW GLOBAL VARIABLES LIKE 'configuration';
> ...
> | configuration | /etc/my.cnf:/home/oli/.my.cnf:command-line |
> ...
>
> Regads Oli
>
> PS: Giuseppe: What do you need the $HOME/.my.cnf for? Why are you not just linking the /etc/my.cnf to
> /home/mysql/data_123/my.cnf?
> (The approach with /etc/my.cnf does only work with one instance per server anyway and then you have to have several
> my.cnf...)
>
> ----Ursprüngliche Nachricht----
> Von: <email address hidden>
> Datum: 29.07.2008 04:31
> An: <email address hidden>
> Betreff: Re: [Bug 252780] Re: drizzle is using MySQL option files
>
> On Tue, Jul 29, 2008 at 4:10 AM, Monty Taylor <email address hidden> wrote:
>> For now, though, and for this bug, I'll just replace references to
>> my.cnf with drizzle.cnf and /etc/mysql with /etc/drizzle...
>>
>
> If you do this change, please keep two option files in the user
> controlled space
>
> * $BASEDIR/drizzle.cnf or $DATADIR/drizzle.cnf
> * $HOME/.drizzle.cnf
>
> access to /etc/drizzle... requires root privileges. We must preserve
> the flexibility of dealing with drizzle in user space.
>
> Giuseppe
>

[Giuseppe Maxia (giuseppe-maxia)]

On Tue, Jul 29, 2008 at 7:48 AM, Monty Taylor <email address hidden> wrote:
> I agree on 1 for the server... There should be one location - not a
> search path of locations that all add together (or not).
>
> However, I _do_ think that 2 locations for client program is good and
> useful - one system wide and one per user (/etc/drizzle/client.conf &&
> ~/.drizzle.conf) ... this is pretty standard across the unix world...
>
> But the many files in a path thing is terrible.
>

Multiple option files are needed to allow a degree of flexibility
* /etc/drizzle.cnf is used by all the instances of drizzle in the box.
* $BASEDIR/drizzle.conf or $DATADIR/drizzle.conf are options for the
specific instance of drizzle

Personally, I don't use /etc/my.cnf (to become drizzle.cnf) because it
is not accessible by a normal user, and if you are not aware of its
existence, you can get many frustrating errors when trying to install
in a non standard location.

The user option file is needed to influence the behavior of client
applications, either built-in or custom made, using the appropriate
[tags].

Please leave the double option file in user space. There are specific
usages for both. The lack of flexibility was one of the reasons for
the fall of MySQL instance manager.

Giuseppe

[Chad Miller (cmiller)]

Rather, have one required config file. Default in /etc/ or SYSTEM_CONFIG_DIR , and may be overridden on command line.

That config file may have an INCLUDE directive (with variable expansion) that refers to other files.

That way, there's no ambiguity; there is one parsing path that is common to all architectures.

[Giuseppe Maxia (giuseppe-maxia)]

On Tue, Jul 29, 2008 at 4:52 PM, Chad MILLER <email address hidden> wrote:
> Rather, have one required config file. Default in /etc/ or
> SYSTEM_CONFIG_DIR , and may be overridden on command line.
>
> That config file may have an INCLUDE directive (with variable expansion)
> that refers to other files.
>
> That way, there's no ambiguity; there is one parsing path that is common
> to all architectures.

This solution has two problems:
1) if the INCLUDE directive points to a path that only root can
access, then you can't (easily) install drizzle in your own path
2) The INCLUDE directive can only use absolute paths, again limiting
the places where you can install.

When deciding the location of these files, keep in mind these constraints:
* the user may not have root access
* the user may want to install several instances of the server, with
different options

IMO, there *must* be two option files:
* the global one (/etc/drizzle.conf) recommended for production and
automated installation
* a customized one ($DATADIR/drizzle.conf) , useful for testing and
multiple instances

[Hartmut Holzgraefe (hartmut-php)]

... and unlike mysqld the global path should not be hard coded but settable via

   configure --sysconfdir

See e.g. MySQL bug reports

  http://bugs.mysql.com/4993 "option '--sysconfdir=/etc/mysql' doesn't work"
  http://bugs.mysql.com/8649 "./configure --sysconfdir=/someplace does not find my.cnf in /someplace"

[Monty Taylor (mordred)]

Hartmut Holzgraefe wrote:
> ... and unlike mysqld the global path should not be hard coded but
> settable via
>
> configure --sysconfdir
>
> See e.g. MySQL bug reports
>
> http://bugs.mysql.com/4993 "option '--sysconfdir=/etc/mysql' doesn't work"
> http://bugs.mysql.com/8649 "./configure --sysconfdir=/someplace does not find my.cnf in /someplace"
>

Agree

[Monty Taylor (mordred)]

Chad MILLER wrote:
> Rather, have one required config file. Default in /etc/ or
> SYSTEM_CONFIG_DIR , and may be overridden on command line.
>
> That config file may have an INCLUDE directive (with variable expansion)
> that refers to other files.
>
> That way, there's no ambiguity; there is one parsing path that is common
> to all architectures.
>

Well, there need to be naturally more than one, as a single file
(/etc/drizzle.conf) presents problems for distributors who split client
and server into different packages (who gets the default file?)

A default location, (/etc/drizzle) which is settable via --sysconfdir,
allow us not only to have different files (server.conf, client.conf,
drizzle.conf, etc) but also allows an easy way to plugins to stick a
config file someplace expected.

Monty

[Monty Taylor (mordred)]

datacharmer wrote:
> On Tue, Jul 29, 2008 at 4:52 PM, Chad MILLER <email address hidden> wrote:
>> Rather, have one required config file. Default in /etc/ or
>> SYSTEM_CONFIG_DIR , and may be overridden on command line.
>>
>> That config file may have an INCLUDE directive (with variable expansion)
>> that refers to other files.
>>
>> That way, there's no ambiguity; there is one parsing path that is common
>> to all architectures.
>
> This solution has two problems:
> 1) if the INCLUDE directive points to a path that only root can
> access, then you can't (easily) install drizzle in your own path
> 2) The INCLUDE directive can only use absolute paths, again limiting
> the places where you can install.
>
> When deciding the location of these files, keep in mind these constraints:
> * the user may not have root access
> * the user may want to install several instances of the server, with
> different options

Both of these should be solvable by making --sysconfdir actually work
properly, and by making "drizzled --confdir=whereever" work, and be easy
to do.

> IMO, there *must* be two option files:
> * the global one (/etc/drizzle.conf) recommended for production and
> automated installation
> * a customized one ($DATADIR/drizzle.conf) , useful for testing and
> multiple instances
>

The problem with DATADIR/drizzle.conf is:

1) putting config files by default in datadir breaks FHS
2) it's got a chicken-and-egg problem - it's tied to either compiled in
DATADIR, or one you set on the command line. If you set --datadir on the
command line, you can just as easily set --confdir instead

So I still think we should have (1) location for the server conf file,
but that it should be easy to specify where it is, so that we meet
guiseppe's need for multiple installs/non-root users.

Monty

[Monty Taylor (mordred)]

I've pushed a patch for this to:

lp:~mordred/drizzle/bug252780

But I'd like to get some feedback/testing on it before I suggest it for
merge.

I've done this:

Renamed the conf files to drizzle.cnf
Removed the bulk of the old search path, replacing with:
  $(sysconfdir)/drizzle
  --defaults-extra-file=<path> (run-time option)
  ~

Turns out, DATADIR/my.cnf is actually not a standard choice anyway,
unless datadir happens to be the homedir of the user running mysql...

Monty

Monty Taylor wrote:
> datacharmer wrote:
>> On Tue, Jul 29, 2008 at 4:52 PM, Chad MILLER <email address hidden> wrote:
>>> Rather, have one required config file. Default in /etc/ or
>>> SYSTEM_CONFIG_DIR , and may be overridden on command line.
>>>
>>> That config file may have an INCLUDE directive (with variable expansion)
>>> that refers to other files.
>>>
>>> That way, there's no ambiguity; there is one parsing path that is common
>>> to all architectures.
>> This solution has two problems:
>> 1) if the INCLUDE directive points to a path that only root can
>> access, then you can't (easily) install drizzle in your own path
>> 2) The INCLUDE directive can only use absolute paths, again limiting
>> the places where you can install.
>>
>> When deciding the location of these files, keep in mind these constraints:
>> * the user may not have root access
>> * the user may want to install several instances of the server, with
>> different options
>
> Both of these should be solvable by making --sysconfdir actually work
> properly, and by making "drizzled --confdir=whereever" work, and be easy
> to do.
>
>> IMO, there *must* be two option files:
>> * the global one (/etc/drizzle.conf) recommended for production and
>> automated installation
>> * a customized one ($DATADIR/drizzle.conf) , useful for testing and
>> multiple instances
>>
>
> The problem with DATADIR/drizzle.conf is:
>
> 1) putting config files by default in datadir breaks FHS
> 2) it's got a chicken-and-egg problem - it's tied to either compiled in
> DATADIR, or one you set on the command line. If you set --datadir on the
> command line, you can just as easily set --confdir instead
>
> So I still think we should have (1) location for the server conf file,
> but that it should be easy to specify where it is, so that we meet
> guiseppe's need for multiple installs/non-root users.
>
> Monty
>

[shinguz (oli-sennhauser)]

> When deciding the location of these files, keep in mind these constraints:
> * the user may not have root access

I understand and fully agree with this concept. But: In the last 2.5 years I have only seen 1 or 2 customers really
had separation between root and dba roles...

Oli

[Giuseppe Maxia (giuseppe-maxia)]

On Wed, Jul 30, 2008 at 5:34 AM, shinguz <email address hidden> wrote:
>> When deciding the location of these files, keep in mind these constraints:
>> * the user may not have root access
>
> I understand and fully agree with this concept. But: In the last 2.5 years I have only seen 1 or 2 customers really
> had separation between root and dba roles...

In production, it is advisable to install MySQL as super user.
Before production, there is development, and QA.
If the only thing I can do for testing is installing as root after
removing the previous instance, I give up the task.
Let's be flexible, for our own safety.

Giuseppe

[Hartmut Holzgraefe (hartmut-php)]

and even before that there's students who can only install things in their $HOME and may even have to have it coexist with other students installations on the same machine ...

[Monty Taylor (mordred)]

datacharmer wrote:
> On Wed, Jul 30, 2008 at 5:34 AM, shinguz <email address hidden> wrote:
>>> When deciding the location of these files, keep in mind these constraints:
>>> * the user may not have root access
>> I understand and fully agree with this concept. But: In the last 2.5 years I have only seen 1 or 2 customers really
>> had separation between root and dba roles...
>
>
> In production, it is advisable to install MySQL as super user.
> Before production, there is development, and QA.
> If the only thing I can do for testing is installing as root after
> removing the previous instance, I give up the task.
> Let's be flexible, for our own safety.

I dunno... I agree with being flexible... and I think we can be through
normal sensible configure/make/make install stuff that obeys prefix and
such...

HOWEVER, I think one of the failings with MySQL was our insistence on
being able to drop a tarball of an installed tree on a box and have it
work. Most people I see (my rants about OSX notwithstanding) are either
using package managers to install on dev and test machines - apt-get,
yum, etc. - or are compiling/installing from source... no?

So, proper ${PREFIX} and ${DESTDIR} support and we should be golden...

Monty

[Giuseppe Maxia (giuseppe-maxia)]

On Wed, Jul 30, 2008 at 7:51 AM, Monty Taylor <email address hidden> wrote:
>
> I dunno... I agree with being flexible... and I think we can be through
> normal sensible configure/make/make install stuff that obeys prefix and
> such...
>
> HOWEVER, I think one of the failings with MySQL was our insistence on
> being able to drop a tarball of an installed tree on a box and have it
> work. Most people I see (my rants about OSX notwithstanding) are either
> using package managers to install on dev and test machines - apt-get,
> yum, etc. - or are compiling/installing from source... no?

You can drop a tarball and have it work. Just use the right tool.
http://launchpad.net/mysql-sandbox

>
> So, proper ${PREFIX} and ${DESTDIR} support and we should be golden...
>

The harder you make it for people to test it, the less tests you will get.
This is an open source project. Meaning that nobody is forced to test it.
If you make things difficult for testers, you throw away valuable contributions.
Please don't make the option file depending on root access. Think of
your potential QA helpers.

Giuseppe

[Monty Taylor (mordred)]

datacharmer wrote:
> On Wed, Jul 30, 2008 at 7:51 AM, Monty Taylor <email address hidden> wrote:
>> I dunno... I agree with being flexible... and I think we can be through
>> normal sensible configure/make/make install stuff that obeys prefix and
>> such...
>>
>> HOWEVER, I think one of the failings with MySQL was our insistence on
>> being able to drop a tarball of an installed tree on a box and have it
>> work. Most people I see (my rants about OSX notwithstanding) are either
>> using package managers to install on dev and test machines - apt-get,
>> yum, etc. - or are compiling/installing from source... no?
>
> You can drop a tarball and have it work. Just use the right tool.
> http://launchpad.net/mysql-sandbox
>
>> So, proper ${PREFIX} and ${DESTDIR} support and we should be golden...
>>
>
> The harder you make it for people to test it, the less tests you will get.
> This is an open source project. Meaning that nobody is forced to test it.
> If you make things difficult for testers, you throw away valuable contributions.
> Please don't make the option file depending on root access. Think of
> your potential QA helpers.

ABSOLUTELY...

I think I've maybe been unclear.

I completely agree that root access should not be needed for conf file
access.

I believe that it should be possible to have a conf file anywhere you
like, even after installation. I believe you will probably have to add a
--config=location/to/config to the drizzled command... but I don't think
that's too much... or?

Monty

[shinguz (oli-sennhauser)]

We are already talking about the solutions...

Suggestion: Let's collect the requirements and weight them. Then we can maybe see, what is the right way to do it?

Oli

PS: Monty: Does launchpad provide some voting facilities?

----Ursprüngliche Nachricht----
Von: <email address hidden>
Datum: 30.07.2008 08:28
An: <email address hidden>
Betreff: Re: [Bug 252780] Re: drizzle is using MySQL option files

datacharmer wrote:
> On Wed, Jul 30, 2008 at 7:51 AM, Monty Taylor <email address hidden> wrote:
>> I dunno... I agree with being flexible... and I think we can be through
>> normal sensible configure/make/make install stuff that obeys prefix and
>> such...
>>
>> HOWEVER, I think one of the failings with MySQL was our insistence on
>> being able to drop a tarball of an installed tree on a box and have it
>> work. Most people I see (my rants about OSX notwithstanding) are either
>> using package managers to install on dev and test machines - apt-get,
>> yum, etc. - or are compiling/installing from source... no?
>
> You can drop a tarball and have it work. Just use the right tool.
> http://launchpad.net/mysql-sandbox
>
>> So, proper ${PREFIX} and ${DESTDIR} support and we should be golden...
>>
>
> The harder you make it for people to test it, the less tests you will get.
> This is an open source project. Meaning that nobody is forced to test it.
> If you make things difficult for testers, you throw away valuable contributions.
> Please don't make the option file depending on root access. Think of
> your potential QA helpers.

ABSOLUTELY...

I think I've maybe been unclear.

I completely agree that root access should not be needed for conf file
access.

I believe that it should be possible to have a conf file anywhere you
like, even after installation. I believe you will probably have to add a
--config=location/to/config to the drizzled command... but I don't think
that's too much... or?

Monty

--
drizzle is using MySQL option files
https://bugs.launchpad.net/bugs/252780
You received this bug notification because you are a member of Drizzle-
developers, which is subscribed to Drizzle.

Status in A Lightweight SQL Database for Cloud and Web: Confirmed

Bug description:
the drizzle server (drizzled) and launcher (drizzled_safe, provided that we decided to preserve it or replace with
something else), and the drizzle clients (bin/drizzle*) are currently using MySQL option files.

/etc/my.cnf
/etc/mysql/my.cnf
$BASEDIR/etc/my.cnf
$HOME/.my.cnf

How to repeat:
===========
(1)
* create a $HOME/.my.cnf file containing the following
[mysqld]
fail-here

* launch drizzled

(2)
* create a $HOME/.my.cnf file containing the following
[client]
database=not_existent

* launch drizzled
* launch drizzle

Why this is serious
==============
options intended for MySQL are not necessarily intended for Drizzle. They must have a separate set of option files.

[Monty Taylor (mordred)]

drizzle no longer reads my.cnf files. It now reads drizzle.cnf files.