Installing Refine Extensions

Introduction

OpenRefine, the framework which Ontotext Refine is build upon, provides a mechanism for extending its functionalities. This extension mechanism is driven by the Butterfly framework. OpenRefine and, therefore, Ontotext Refine extensions, are Butterfly modules. However, no knowledge of Butterfly is necessary.

A set of default extensions are shipped together with the distribution, located in the extensions subdirectory. When you want to add your own extensions, you can place them anywhere, as long as you point Butterfly to it.

The extensions have to be placed on the same filesystem as Ontotext Refine.

Compatibility Notice

Many of the extensions which are developed for OpenRefine are out of date. You can check the current version which Ontotext Refine ships with from the “about” menu of the application.

Warning

Most outdated extensions will not run correctly, some would completely crash the application.

The list of extensions which we have verified to run with Ontotext Refine are:

  • database
  • jython
  • pc-axis
  • phonetic
  • wikidata
  • rdf-transform

Warning

The gdata extension has a problem related to the way in which iframes are managed and has been disabled for this version of Ontotext Refine. We would like to incorporate it in the future.

Developing New Extensions

You can use the tutorial provided by OpenRefine. Keep in mind that due to the specific server implementation of Ontotext Refine, extensions which add new popup windows will not function properly.

Extensions Directory

The easiest way to set up the extensions is to store them within the extensions directory. The path to the directory is relative to the application source, lib/refine/extensions.

Care should be taken when you are modifying the extensions directory as it is part of the application source.

Module Path Property

Setting up modules can be as easy as setting a new butterfly.modules.path. This will not override the default paths.

You can also set up multiple paths if your modules are not placed within the same location.

Alternatively, you can set up a list of directories in the same property: -Dbutterfly.modules.path=modules,../../extensions,/path/to/your/extension

Deployment-specific Configurations

Onto Refine Desktop

The Ontotext Refine native application has a few caveats, but the general approaches to setting the extensions directory still apply.

The extensions directory in the native app is ${ontorefine.dist}/app/lib/refine/extensions.

You can set up the butterfly modules path in the .cfg file, at ${ontorefine.dist}/app/Ontotext Refine.cfg. Add the line java-options=-Dbutterfly.modules.path=modules,../../extensions,/path/to/your/extension to the bottom of the file.

Alternatively, you can conveniently set up the butterfly.modules.path property in the UI, from the Settings window. If you have multiple module paths, you need to comma-separate them within the same property, as described above.

Finally, you can use the “Set Extensions and Restart” button. This opens your file explorer. You can select your extension folder from here. This sets the butterfly.modules.path property.

The drawback of this approach is that you can only set one path.

Standalone Distribution

The general approaches outlined above are also available for the standalone distribution.

The extensions directory in the native app is ${ontorefine.dist}/app/lib/refine/extensions.

You can use the –extensions startup argument to point towards the module paths, e.g. ./ontorefine.sh –extensions modules,../../extensions,/path/to/your/extension

You can set up the butterfly modules path within the environment variable ONTOREFINE_JAVA_OPTS.

Alternatively, you can directly pass the property by using the -D flag, e.g. ./ontorefine.sh -Dbutterfly.modules.path=modules,../../extensions,/path/to/your/extension.

Docker Distribution

When running from Docker, the same ways to set up the extension modules path as in the standalone application apply.

The only difference is that the modules need to be passed inside the container as a volume.

docker run ontotext/refine -v /path/to/your/extension:/container/internal/custom/extension -e ONTOREFINE_JAVA_OPTS=-Dbutterfly.modules.path=modules,../../extensions,/container/internal/custom/extension

Example Extension Setup

Here are the steps for setting up an extension with the native application:

  • Set up your extension in a folder. The file structure needs to be the following:
extension-folder/
    extension-name/
        module/
  • Set up the butterfly.modules.path to point towards the extension-folder.
  • Restart Ontotext Refine to pick up your new extension.

Note

Butterfly searches for extensions recursively in sub-folders. If you add a folder with many sub-directories, the startup process will be slower. It’s also possible to get unexpected modules loaded. Therefore, we recommend a relatively flat structure.