Additional fields vs. Custom fields for Statistical fields and codes

The purpose of this post is to start a discussion about whether Statistical fields regarding Users should be a Custom field or an Additional field. What follows comes from my notes during the UM-SIG meeting earlier today.

Points in favor of using additional fields for statistics:

• Statistical categories are already present in the Inventory app, and we might be able to use the same structure in the Users app.
• Additional fields would be in the LDP. We can write reports against the LDP. Reports using Custom fields might be trickier, because they are not in the LDP.
• Many institutions need Statistical fields, and we’re all planning on configuring Custom fields separately. Perhaps we could all make use of this structure by having it as Additional fields, so we don’t wind up duplicating each others’ work.
• Additional fields can be migrated. The user import can’t manage Custom fields at this time.

Points in favor of having Custom fields for statistics:

• Custom fields can be created today. JIRA tickets asking for Additional fields are not MVP.
• Custom fields are more flexible than Additional fields, because Additional fields require a controlled vocabulary.

Questions to help us focus the discussion:

• Should all of the Statistical fields be repeatable within a User record?
• What requirements should we have for reports using Statistical fields? We should be specific about what we need, not about how we go about getting these reports.
• What requirements should we have for Statistical codes?
• Is the model for Additional fields as it exists in the Inventory app a workable model for the Users app?

There was also some followup discussion here in Durham about tags – there are underlying permissions in the tags module that could give us granularity to use them in ways we’d want to for statistical needs. We’d have to write a feature request (or more) for that, but if we’re going to need features anyway… I don’t know if we’d want to reconsider.

It seems like if a number of implementing sites have need for statistical categories for users, it would be much cleaner to do it as a built-in field in UM, rather than leveraging the custom fields.

In terms of getting it done, I wonder if this is a small enough lift that an implementing site with on-staff developers could take it on? Perhaps with a little coaching. Seems like a potential learning project for someone who wants to grow into more complicated projects.

• Should all of the Statistical fields be repeatable within a User record?

I personally don’t think statistical fields should be repeatable, as I think that invites double counting of use. I like our current approach of assigning a primary statistical category when users have multiple relationships (e.g., it would be misleading to characterize me as an alumnus even though I have 3 degrees from Chicago because I am employed by the Library with an academic appointment and most of my use stems from that relationship).

I think the first question we should answer is this: What do we need from statistical reports? Then, we can ask which option (Additional fields, Custom fields, Tags) suits the requirements. Aside from the fact that Tags are available now, what is it about tags that give you what you want?

Cornell would want an Additional field that’s repeatable. A user needs to be able to have multiple stat codes. For reporting, we use the stat codes in two ways: as selection criteria for generating reports on the demographics that the code represents, and as a data element on reports that include patron demographics.

The stat codes must be able to be updated in an automated way from an external system (via the patron load); and must be editable locally – e.g., for guest borrowers, and internal processing cards like Borrow Direct, bindery, faculty routing, etc. The Inventory UI model might work for us, as our non-departmental stat code list isn’t that long.

@tod - I’m sure @nielserik (Index Data) - who was the one who did the work on implementing Statistical codes, and Statistical code types in Inventory (instance, holdings, item) - would be able to coach anyone, who were to pick up the task implementing something similar for User records

I don’t know that it is misleading. It’s another affiliation that may grant you additional services in some libraries (special events admission, maybe?) or be desirable to capture for specific kinds of outreach. It doesn’t control main services, perhaps, but it’s still valuable. But, then the library would probably have a statistical code for primary relationships, and then a statistical code for secondary relationships that is repeatable.

One of the big appeals is features that are currently in queue to give permissions on tags and individual records, that we would not have on statistical codes. It’s UXPROD-258 and UXPROD-489. So you could then push values to a tag from an outside data source and make them non-editable within the GUI - providing some assurance that the data would remain consistent.

UChicago staff will need to manually edit statistical categories within the GUI for certain users when their primary relationship changes, and there is a large cohort of users that will not be coming in on a data feed.

Not certain how tags would work as statistical categories when that category changes.

Here’s a hypothetical for an institution that has one primary statistical category for each user: A student in a masters program has had an account for the last year with a statistical tag of “Student-Graduate”. The student graduates, becomes an employee, and now comes across in a new data feed as having staff privileges. Does the import process simply add a new tag of “Staff” in addition to the existing “Student-Graduate” tag? If it did, how would anyone know which tag was the current category? Reporting would almost immediately become inaccurate as bad statistical tags stacked up with each data feed.

Since that’s not helpful, let’s say we want the import process to remove the previous tag before adding a new one. How does the import process know which existing tag on that user’s account is the tag for the user’s statistical category so that it can remove it?

Or, what if a staff member is manually editing a record, adds a current statistical tag, but forgets to delete the old statistical tag? Again, bad tags stacking up.

For schools with multiple statistical categories per user, how do you use tags to identify a primary affiliation?

I haven’t heard one actual advantage to using tags instead of dedicated fields (additional or custom) for statistical categories.

Sure, that makes sense. So in that case, you could apply tag permissions in the way that works best for your institution.

I’m not arguing per se for one solution - but I do think it makes sense to work through the options.

I think there could be potential advantages in terms of ease of use for smaller institutions; the fact that tags are already implemented and part of the LDP (at least as I understand it.)

Having said that, your other points are valid, though I’m not sure I understand how history of statistical categories would be tracked over time. @maura I think that’s a good question to bring back to the UM SIG next week.

I’m not sure I understand how history of statistical categories would be tracked over time.

What do we need from our statistics? Do we need an audit trail, or a history of statistical codes that are/had been applied to a user? Or does that not matter? I’m not one to speak on this, so I hope that others will chime in.

Well, one need I could see is for a statistical code to be part of a loan record so that when the loan is anonymized, you could still retain that demographic data about the borrower. I know in our Aleph instance, both the patron group and the statistical code are retained after anonymizing.

I assume patron group remains on the loan record when anonymized, but I actually don’t know that for sure. @ecboettcher might know.

That would imply a requirement that the patron statistical code be recorded in the loan at the time of loan. Is that a need that we have today, or a possible future need?
I ask because if it is a real, concrete need, then it argues for a field for statistical category as a baked-in part of User Management, rather than using a custom field or what not.

I agree. One of our libraries would like to have a flag for patrons who asked for access to some databases which are not accessible for all library users in general. The library staff will create accounts for them and per Statistical Code they could search resp. count the patrons with acess to database X or database Y.

So far I have considered tags as data fields that are used more or less only temporarily. For example, as markers for data records that you want to edit again later.

Hi @tod - I think it does argue for a built in feature, because that data is going to be important after anonymization. But whatever code built for it would need to be able to be consistently named and that’s not really something I think that fits the statistical category model. So it might even be another field that we’ve identified as a need (sob, no one tell @pattywanninger)