More great news on the ever-growing SobekCM community and ever-improving SobekCM Open Source Software! The information below is copied from the SobekCM-discuss Google Groups Discussion Group. Huge thanks to everyone who contributed code, testing, and documentation to make this release possible!
I’m particularly excited about the new user permission reports and collection change logs, which are essential tools for working at scale. And, I’m still looking through this long list of updates to see what else to be excited about!
Also, come to THATCamp-Gainesville this Friday, April 24, 2015, to meet many of the developers, testers, and users who contribute to SobekCM, including many members of the new SobekCM Advisory Board!
Overview
We are proud to announce the release of Version 4.8.7 of the open-source digital repository SobekCM, along with a new MSI installer for simple installation of this version (and much of the help related to the software) at http://sobekrepository.org/
There were numerous updates released in the first quarter of 2015, culminating in the release of the new installer for version 4.8.7. Development focused on general performance and design improvements, including embracing an international content delivery network for many of the static files used by the system and other changes to result in a large performance gain for real users. Customization support was added for MARC21 generation and OAI-PMH metadata formats, from new system-wide settings to ability to apply a XSLT to the MARC21 after generation. Development focused on improving the ease of online collection management, editing the look and feel of the site, and other administrative options with many new features available to authenticated and permissioned users. And finally, the system underwent continuing architectural revolutions with a completely rewritten modular builder service and with an increasing acceptance of a REST-based microservices approach.
For a complete list of the many changes, please view the list of updates from the first quarter here: http://sobekrepository.
You can view the open source code on GitHub as well at: https://github.com/
We welcome all interested contributors, or any questions you may have about the software or the installation process.
Thank you to all the other contributors, both those who contributed code as well as those that provided guidance and helped us continue to improve upon the administrative features of the system.
Update Document
Updates to SobekCM in first quarter 2015
There were numerous updates released in the first quarter of 2015, culminating in the release of the new installer for version 4.8.7. Development focused on general performance and design improvements, including embracing an international content delivery network for many of the static files used by the system and other changes to result in a large performance gain for real users. Customization support was added for MARC21 generation and OAI-PMH metadata formats, from new system-wide settings to ability to apply a XSLT to the MARC21 after generation. Development focused on improving the ease of online collection management, editing the look and feel of the site, and other administrative options with many new features available to authenticated and permissioned users. And finally, the system underwent continuing architectural revolutions with a completely rewritten modular builder service and with an increasing acceptance of a REST-based microservices approach.
In total, we saw the release of eight versions. The code base saw 140 individual commits from users within four institutions.
The new version can be downloaded at: http://sobekrepository.org/software/download
Below is a look at the individual changes from the first quarter of this year:
General Performance and Design Improvements
- Content Delivery Network (CDN)
- Updated the way all the static version-specific files are linked throughout the system, affecting all standard images, javascript, stylesheets, and included libraries
- The static files to use are read from configuration files, allowing a user-specific configuration file to override just one or many of the file references
- By default, all these files are served from a content delivery network, increasing speed and performance for end users by 25-40%, and reducing the number of hits against the server by close to 40%
- Benefit of the content delivery network will greatly improve performance from international users, as the CDN has over 30 nodes internationally
- Included Libraries (both javascript and stylesheets)
- Re-examined all included libraries and added some more logic to only include some libraries when actually needed
- Minimized all included libraries to improve performance
- Design improvements
- Update the style of the messages for errors and successes across all admin screens to make the message and the severity more obvious
- Changed the design for all the system admin screen, removing the gray title box and adding the icon for that task to the screen, to allow users to better associate the icons with the tasks
- Updated most the internal and aggregation curator screens to use a similar look and feel as the main admin screens
- Contact Us form
- Completely recreated the contact us form
- Can customize the form to have any number of elements, including hidden elements by using the configuration file
- Can additionally customize the form for individual aggregations to have aggregation-specific forms
- Email configuration
- Can now choose to either use the Database Mail system, or to email directly via SMTP
- Can now set the FROM email address and display name in the system-wide settings
- Centralized all emailing to use the single Email_Helper class
- A number of small improvements and bug fixes for emailing item and collection information
- HTTPS support added
- Changes to registration process
- URL changed to /register, rather than duplicating /my/preferences to make cleaner URLs when sending them via email
- If a user enters their name with all caps or all lower case, the name is automatically title-cased
- Help now defaults from http://sobekrepository.org/sobekcm, rather than UFDC
Aggregation Viewer Updates
- Updates to the public aggregation-level main menu
- If no thematic headings exist, the variety of home pages ( list, brief, tree ) are suppressed
- If a child static page is selected, the parent menu item is now highlighted correctly
- Restored Tree View on the main instance home page to work correctly, and for the collapse and expand all features to work correctly without going back to the server
- Aggregation Item Count – Corrected an exception thrown when there were no items in the aggregation
Item Viewer Updates
- Added more JPEG2000 support, especially for separate IIP image server, by adding new system-wide setting – Image Server Root – to override the defaults
- Updates to the PDF viewer
- Allow PDF title to contain single and double quotes
- Fixed a bug for PDF delete function
- Updates to the HTML viewer, which allows a HTML file to be embedded as one of the item views
- Restored the HTML item viewer
- If a HTML/HTM file is referenced by a HTML viewer and in the METS, it no longer triggers the downloads tab.
- Updated the way viewers are added back to the digital resource object from the database to be more clear and easier to update and support later changes
- Citation Viewer updates
- Switch to using the main thumbnail in the citation viewer for better performance
- If the rights statement has a URL, it is correctly displayed as a link.
Metadata and OAI-PMH
- MARC21 Generation
- Moved several tags into the standard MARC tags, to always be included.
- Collections are now listed in the 830 field
- Thumbnail is included in the 992 field
- Added ability to customize certain MARC fields across an instance through system-wide settings
- Cataloging source code – for the 040 field, was defaulting to FUG
- For reproductions only (not born digital)
- Agency responsible for reproduction, for the added 533 |a field
- Place of reproduction, for the added 533 |b field
- Location code, for the 852 |a field
- Set XSLT file to be applied to the MarcXML records after creation
- Added extended Dublin Core (and Duke specific) tags to the Dublin Core reader class
- If a TYPE does not map to a standardized type, it is mapped to Genre automatically
- Accession number updates
- Accession number now saves as an item group external record number
- Accession number saves correctly in the metadata searching tables
- Updates to metadata templates – Template XML files are now divided into two main folders – default and user
- Default includes the templates that come with the system and may be changed during an upgrade
- Templates put in the user folder override the template in the default and will not be changed during upgrades
- Updates to metadata template elements
- Holding location and source institution fields now work correctly without specific aggregation rights
- Updates to a controlled rights statement – currently utilized by UF
- OAI-PMH
- Added new configuration file that allows any metadata type to be served as OAI-PMH
- External libraries can easily be referenced to override the standard creation of metadata for OAI-PMH
- By default, MarcXML is included as an OAI-PMH metadata option
- Updated resumption token to work correctly – did not work in certain situations
- Moved several tags into the standard MARC tags, to always be included.
Updates to Collection Management / Curation
- Created Add New Collection Wizard to which steps an administrator or curator through the process of adding a new collection, with the following new capabilities:
- Aggregation type of the new aggregation is suggested based on the parent
- External link is hidden except for institutional type aggregations
- Short name is automatically populated from the name when leaving the name field
- Flag to include in the parent collection is hidden when the parent is the top-level home
- Can add a new thematic heading at the same time the new aggregation is added
- Can upload the collection banner, or choose to let the system auto-generate the banner
- Can upload the collection button, otherwise the system default is used
- Better default values in general provided
- Added a new aggregation admin/curator home page within the aggregation, available from the internal header, with access to all the aggregation-level management options.
- Added new Users Permissions report which shows the list of all users that have special rights over the current collection. This report is accessible from the aggregation admin home menu and shows rights directly given to users, or assigned through user groups.
- Added a new Collection Change Log which shows all the updates to the design files, settings, uploads, and home pages for each aggregation. This does not include changes to the digital resources, just changes to the collection itself. This report is also accessible from the new aggregation admin home menu.
- Major updates to the Single Aggregation administration screens
- Configuration files are now backed up when editing through the online interface
- Added a new Uploads tab which allows user to manage uploads
- Uploads can be deleted
- Can upload images (several at time), for inclusion in the home page or child page text
- Can upload documents to associate with the aggregation (such as tracking documents or the source files for the graphics used)
- Added a new SAVE button to save the changes and continue editing, and changed existing button to SAVE & EXIT.
- Some tab maintenance, such as hiding the Highlights tab if unused and moving the commonly used Appearance tab up to the second position.
- Hid the external link field if the aggregation is not an institution-type collection
- Added help to each of the help buttons on all the tabs of the aggregation admin screen
- Added an image of each search screen to view when selecting which searches to include in an aggregation
- Can now save multiple web skins to an aggregation
- Added ability to delete unused banner images
- Viewing the source files ( most notably the CSS ) now opens in a new window
- Top-level home page now supports multi-lingual versions, same as all the child aggregations
- Bug Fixes:
- Corrected issue where selecting an eighth facet was causing the system to fail to show any results or facets
- All changes from the admin screen take effect immediately now
- Subcollections now show in the SubCollection tab, even if hidden or inactive.
- Removed FULL from the possible search result views – deprecated
- Corrected the display of links for the source files (was white on white)
- Aggregation (static) child pages
- Can now edit the home page html for all the child pages, including the browse-by version
- Putting child pages on the main menu under existing child pages now works, allowing hierarchical child page navigation directly from the publicly accessible aggregation men
- New Aggregations
- Automatically generated home page text updated to include a link to the parent collection and the description entered when the aggregation was added
- An email is now sent to the system administrators when a new aggregation is added to the system (can be controlled via new system-wide setting )
- Aggregation permissions are copied from the parent collection to the child collection during creation
- Exhibits now support subcollections, just like all other aggregations
Internal Views and System Administration
- Improved the Collection List internal view – available to power/internal users
- Existing list view updated to allow filtering, sorting, and searching
- Added a new Collection Hierarchy view that shows ALL of the collection hierarchy (including inactive and hidden collections) in a collapsible tree view
- Updated the system/portal admin main menu
- It is now a deeper categorized system, with more options
- Common tasks appear at the top, and include the most commonly used tasks
- Updated the system/portal admin home screen
- Brief view includes a description of each option
- Classic view is similar to the old version display
- Categorized and Alphabetical display icons with just the title
- Recreated many of the icons used in the admin home screen
- Updates to the Users and User Groups management
- Fixed issues reported when viewing user/user groups and individual permissions
- Updated the user group management screens to work more like the user management screens
- Added ability to delete user groups that are not attached to any users
- Validated all permissions changes in both the standard and detailed permissions modes
- Added new User Permissions Report at the top-level, accessible to system and portal admins
- Displays users with permissions provided, either linked directly to the user or through user group membership
- Global list – shows all the users with top-level permissions (i.e., system admin, can delete all, can edit all, etc.. ) displayed as lists under each permission type
- Global table – shows users in a tabular format with filtering, sorting, and searching that have the same top-level permissions as above
- All Aggregations – lists every user that has specialized permissions on aggregations, along with a list of all the aggregations each user has permissions for. Permissions granted are displayed separately for those individually provisioned and those assigned through user group management
- Single Aggregations – lists the details of permissions granted to users and user groups for a single aggregation
- Can Submit Items – lists all users that have submittal rights and includes which templates and default metadata sets users are linked to.
- Aggregation Management (top-level, under the portal/system admin options)
- Removed the DELETE link from the ALL collection, as it cannot be deleted
- Updated the list of aggregations
- All of the aggregations are now displayed simultaneously, but the list is now filterable, sortable, and searchable
- Added columns for the parent collection codes and flags to indicate if the collection is active and if the collection is on the top-level home page
- Collection codes for institutions now start with a lower case ‘i’, since the upper case was not always clear
- Users are encouraged to use the Add New Collection Wizard here for new collections, but the old system to add (without uploading banners and buttons) remains, with some modifications to better support bulk adding of aggregations.
- Deleting aggregations (from Aggregation Management and Single Aggregation Admin screens)
- Corrected error message incorrectly reporting not all folders could be deleted, when they really were
- Deleting an aggregation no longer fails if all children aggregations are also deleted
- Adding an aggregation back that was previously deleted now works correct
- Updates to the System-Wide settings
- Added and removed a number of settings (as noted in this documentation)
- Settings viewer now handles deprecation of setting categories easily
- Added new isHosted flag to the main configuration file which removes some system admin features and reserved them for a new class of admin, Host Administrator
- Updates to the Web Skin management screen for system and portal admins
- Checkbox to copy files from the current skin to the new web skin starts checked now
- If the override banner flag is not set, the banner link input is hidden during input (and editing)
- Removed strange Override Header/Footer option for web skin – it now always overrides them
- Can now delete an unused web skin
- Changed the option to edit web skins to use the new single web skin management screen
- Added new online web skin management screens to allow online editing of a single existing web skin
- Can edit the basic information about the web skin (previously available in a pop-up from the main web skin management screen)
- Can add support for new languages to the web skin and delete support
- Can edit the headers and footers associated with the web skin for each language
- Can edit the CSS stylesheet at the web skin level
- When making changes through the new online interface, backups are kept for any files that are changed.
- Can upload images (several at time), for inclusion in the headers, footers, or CSS
- Can upload documents to associate with the web skin (such as tracking documents or the source files for the graphics used)
Installer
- Created new MSI to install version 4.8.7, and upgrade database from previous versions
- Simplified the process of installing a new instance – at least for non-production testing
- Signed MSI with certificate for authenticity
- Installs the SobekCM Builder scheduled task now
SobekCM Builder
- Additional support for a single builder running over multiple instances
- Builder modularization – Builder has been completely rewritten as a modularized workflow with complete ability to customize throughout the workflow. (Technical white paper to follow)
- Can also reference external libraries to add your own custom steps into the builder
- Builder as Windows Service – Currently testing the builder as a windows service, rather than a scheduled task
REST-based Web Architecture
- Divided the former SobekCM_Library project into a number of projects to support better separation between the HTML generation routines and classes and the data access/update classes.
- Added new SobekCM Engine Microservice handler and SobekCM Engine Client classes
- Engine is a loose collection of microservice endpoints, defined in XML
- Client class is also based on a different XML file, so external endpoints could be utilized if desired
- Performed performance testing and completed inclusion of two message types
- Serves JSON through the very quick JIL library
- Serves protobuf messages through the even quicker protobuf-net library for inter-process communication
- Handles POST as well as GET REST request/endpoints
- Endpoints can be restricted by IP address
- Exposed a large number of aggregation and web skin specific endpoints via JSON
- Optimized aggregation and web skin classes for better JSON and protobuf generation.
- Began using much of the final classes, without depending on the REST API code
RT @laurien: SobekCM, Updates in First Quarter 2015: More great news! http://t.co/9bTtRFViRp