Use actor.org_lasso for circ and hold policies.

[Jason Stephenson]

In our consortium, we would like to be able to use actor.org_lasso and actor.org_lasso_map in config.hold_matrix_matchpoint and config.circ_matrix_matchpoint with the goal of having fewer matchpoints and to possibly increase the flexibility of configuring matchpoints.

To that end, I have identified the following, incomplete list of changes:

1. Add flags to config.global_flag:
    a. policy.circ.use_org_lasso
    b. policy.hold.use_org_lasso

2. Modify action.find_circ_matrix_matchpoint to use actor.org_lasso and actor.org_lasso_map when policy.circ.use_org_lasso is enabled.

3. Modify action.find_hold_matrix_matchpoint to use actor.org_lasso and actor.org_lasso_map when policy.hold.use_org_lasso is enabled.

4. Add UI in staff client to edit actor.org_lasso and actor.org_lasso_map.

5. Modify staff client to use actor.org_lasso when editing config.circ_matrix_matchpoint when policy.circ_use_org_lasso is enabled.

6. Modify staff client to use actor.org_lasso when editing config.hold_matrix_matchpoint when policy.hold_use_org_lasso is enabled.

7. Add trigger code to create actor.org_lasso and actor.org_lasso_map entries for newly created actor.org_units when either flag is set.

8. Add/edit actor.org_unit update trigger to change actor.org_lasso.name if actor.org_unit.name changes.

9. Add cascading delete on actor.org_lasso_map and actor.org_lasso when an actor.org_unit is deleted.

This is a WIP, so doubtless there will be more additions to the list above. I've got a two page list of other code that has to do with these tables, so I still have to go through that.

Hopefully, I'll start adding patches next week.

[Jason Stephenson (jstephenson)]

Problem: I'm not sure that this can be made optional.

config.circ_matrix_matchpoint and config.hold_matrix_matchpoint both have columns that reference actor.org_unit.id. There's no way in Pg that I'm aware of to make a column reference two columns in other tables. For instance,

copy_circ_lib INT REFERENCES actor.org_unit (id) OR actor.org_lasso (id) DEFERRABLE INITIALLY DEFERRED,

is not valid syntax.

Since the columns can only reference 1 foreign key, EITHER the change will need to be non-optional and the matchpoint tables' columns changed to reference actor.org_lasso OR new columns will need to be added to the matchpoint tables and those used and populated instead when the appropriate global_flag is enabled.

[Brandon Uhlman (branflakes)]

As opposed to ripping apart the existing org-unit based circ infrastructure and rebuilding it in your image,what about creating a separate interface to create and manage lasso-based circ rules, and having that piece appropriately manage the underlying OU circ rules.

In other words, config.circ/hold_matrix_matchpoint become more like materialized views of config.lasso_circ/hold_matrix_matchpoint, which of course, doesn't exist yet. Done correctly, it might even allow a combination of lasso- and OU-based rules to exist together.

This is just the outcome of a little thought experiment I had when I read your bug report; it might not be practical at all.

[Jason Etheridge (phasefx)]

I remember at one point we were using a scheme where the absolute
value of a negative id in an org field would refer to an org lasso.
If we're willing to lose the referential integrity for the org_unit
column, I bet we could adjust the SQL in
action.find_circ_matrix_matchpoint to accomodate that. Alternately,
we could make the column nullable and add an explicit org_lasso
column.

[Jason Stephenson (jstephenson)]

Brandon, if I understand you correctly, you're suggesting just a UI to manage the circ and hold policies using org lassos, but when the matchpoints are inserted, the org_lasso will be expanded into the constituent org_units by a lookup in org_lasso_map and a matchpoint inserted for each constituent org_unit? That would work for creating the matchpoints, but I'm not sure it would help in maintaining the matchpoints. Unless the org_lasso-based rules are stored somewhere, they'd be lost in conversion, so you couldn't update the matchpoints in the same way. Every single matchpoint would then need to be touched for changes to be made.

Also, the above does not stop the explosion of matchpoints that I see happening when we migrate. I've spent a great deal of effort to trim our circulation and holds rules down to 1,500 or so from over 20,000. I do NOT want to go back to that. The way I see it, the fewer the matchpoints, the less chance of human error and the easier it is to maintain. (Now, that I think of it, I'm pretty much only using the equivalent of lassos with holds policy in our current ILS. The circ rules are going by individual location codes and not using groups.)

(Yeah, I know: force them to have sane policies, don't let them change their policies, but that isn't how it works here. We're composed of 35 autonomous units who decide their own policies and I need a system that accommodates that in a reasonable manner. Not that Evergreen isn't reasonable already, I'd just like to do some things where lassos would be ideal.)

[Jason Stephenson (jstephenson)]

In reply to Jason's comment #3:

I think I'm going to want to have org_lassos in more than just the org_unit column, at least for hold_matrix_matchpoint.

Using absolute values of negative numbers seems a bit kludgey to me. I don't like relying on too many magical values, particularly in something that is aimed at being distributed to others.

I also don't want separate columns for org_lassos and for org_units. I'd like one set of columns that can do either, but at this point I don't see a way to do that without breaking referential integrity.

I started writing some code yesterday with a mind to just breaking everything as it exists today. That is, I coded some trigger functions to maintain a set of default lassos for org_units as they are added and updated. I was going to change the referential integrity on the matchpoint tables to point at actor.org_lasso instead of actor.org_unit, etc., etc. (I didn't get that far, but I did get the triggers written.)

That causes breakage of course for anyone who is already using org_lassos (and possibly some people who aren't but are using in-db circ and holds policy). For it not to break existing matchpoint, the triggers would have to guarantee that there is a one-to-one correspondence in name and id of an existing org_unit and an org_lasso. That is, every org_unit would need to have an org_lasso with the same id. Otherwise, existing matchpoints would not match correctly or would need to be altered with the org_lasso ids. Completely switching to org_lasso isn't exactly desirable because of the high potential for breakage of existing installations.

IF I do this thing, it may very well end up being patch set that is maintained outside of mainstream, but time will tell.

[Jason Stephenson (jstephenson)]

Adding a new pair of tables for circ/hold policy groups might work if triggers were used to maintain a default set as mentioned previously. Just a thought.

[Jason Stephenson (jstephenson)]

That won't work with numeric ids either. Even with triggers at some point, the group and org_unit ids will get out of sync.

The keys for the groups would need to be strings.

This is no good, forget it ever existed.

[Mike Rylander (mrylander)]

I see this is now marked as Won't Fix, but for the next attempt (and I think there should be another attempt at attacking this, at least to the point of ruling it out conceptually, or proving it's useful and possible) I'll provide some context here (which many readers already understand, but has not necessarily been written down, and not in this thread in any case).

We need to step back and consider just what it is that the matrix tables and the functions that use them are doing, what environment they run in, and what state and context they (the functions) have access to.

Generally, the tables define a decision map between the current state of some input (who, what, where, status, age, etc.) and the expected action to be taken (allow or not, how long, at what cost).

First I'll look at the circ case which, while somewhat more complicated in terms of the input and output, is less magical -- it is a direct mapping.

So, we have an org_unit, which defines the owner /of the matchpoint rule/ and /not/ the owner of a copy/item or user or anything else. We also have a grp, which must have a self-or-ancestor relationship to the profile group of the user to whom the item will circulate. The other input fields (circ_modifier, marc_type, marc_form, marc_vr_format, ref_flag, juvenile_flag, is_renewal, usr_age_lower_bound, usr_age_upper_bound) identify the amount of like-ness there is between the rule and the item to potentially circulate, the user to which we will potentially circulate the item, and in the case of is_renewal, the type of potential circulation.

Given that, in what way would the OUs inside an org_lasso contribute to the process of selecting a circ matchpoint? I see two options. Either the lasso can take the place of the org_unit column (though not physically, more in a minute), or the lasso can be used to refine the like-ness of the rule to the input.

In the former case, the effect would be to allow a single row in the circ matrix table to act as a stand-in for an identical row for /each and every OU within the lasso/. Following this a bit further, this would allow a central administrative body to create a set of mutually exclusive rules from which a specific library could choose (i.e., a set of DVD-circ_mod rules) by requesting they be included in the appropriate lasso. From an administrative point of view it would allow for fewer per-OU matchpoints and fewer total matchpoints. This administrative body need not be consortia-wide, of course -- each "system" could maintain its own lasso-based rules and local lassos.

In order to implement this, we would simply make org_unit NULLable, provide a new NULLable FKEY column called org_lasso, and use a CHECK constraint to make sure that only one or the other is in use on any given matchpoint row. Then adjust the query looping logic to take both into account.

But what about the latter (like-ness) case? It does not look to be mutually exclusive with the first. And, in fact, would be simple to implement by providing usr_home_lasso, copy_circ_lasso and copy_owning_lasso columns that act in the same way as copy_circ_lib and copy_owning_lib do (in trunk) today. This would allow one to w...

[Jason Stephenson (jstephenson)]

After reading miker's comment, I've decided not be such a perfectionist and to give it another go. I'll leave this in incomplete status until I think it is ready for integration.

I don't expect this to be ready for 2.0, but maybe for 2.1. I'll do my work in a copy of trunk and will keep my code synced with mainline as much as possible.

I like the second option specified by miker. That's the one that I think will save us on number of rules and spare some maintenance headache.

[Jason Stephenson (jstephenson)]

Given changes in the circ matrix, etc. since this bug was filed. I am not touching it. Since I opened the bug with the intention of providing code, I believe I can say "Won't Fix" without a doubt.