• ELINKAPI - A Python Interface for E-Link 2.0
  • Open Questions
  • Table of Contents
  • Introduction
  • Installation
    • Importing the Package from test.pypi.org
    • Importing the Package from Production PyPI
  • Examples
    • Using the API connection
    • Creating a New Record
    • Seeing Validation Errors on Exception
    • Processing Status of Submitted Records
    • View Revision History
    • VIew Record at Revision
    • Adding Media to Record
    • Removing Metadata Records
    • Removing Media from a Record
    • Compare Two Revision Histories
    • Searching and pagination
  • Method Documentation
    • Configuration
    • Records
      • Revisions
    • Media
  • Classes
    • Record
    • Query
    • Organization
    • Person
    • Identifier
    • Related Identifier
    • Geolocation
    • Media Info
    • Media File
    • Revision
    • Revision Comparison
  • Enumerations
    • AccessLimtation
    • JournalType
    • ProductType
    • PAMSPatentStatus
    • PAMSProductSubType
    • PAMSPublicationStatus
    • WorkflowStatus
  • Exceptions
    • UnauthorizedException
    • ForbiddenException
    • BadRequestException
    • NotFoundException
    • ConflictException
    • ServerException

This module is setup to mimic the E-Link 2.0 API Endpoints (API documentation found here) and allows for you to quickly get up and running submitting Records using Python.

1. Install the package, but don't grab the dependencies (pip will attempt to grab everything from the test server, which we do not want): pip install --index-url https://test.pypi.org/simple/ --no-deps elinkapi
2. Now install the other dependencies: pip install elinkapi
3. Or install them separately: pip install requests pydantic urllib3==1.26.6
4. Access the E-Link connector via from elinkapi import Elink and creating an instance for use with your API key: api = Elink(token="Your_API_Token")
5. API classes are accessible using from elinkapi import Record, etc.
6. Exception classes generated by the API are accessible using from elinkapi import exceptions then catching appropriate exceptions.BadRequestException and the like.

1. Install the package: pip install elinkapi
2. Access the E-Link connector via from elinkapi import Elink and creating an instance for use with your API key: api = Elink(token="Your_API_Token")
3. API classes are accessible using from elinkapi import Record, etc.
4. Exception classes generated by the API are accessible using from elinkapi import exceptions then catching appropriate exceptions.BadRequestException and the like.

In order to use the python API library, it is generally required to create an instance of the "API object" to interact with. Following examples will start out with this, but more details will be provided in this section.

The API instance sets the target URL value (that is, the desired E-Link service) to connect to; by default this will be the production E-Link service. For testing or pre-release Review or Beta access, specify a target to the review E-Link service. Please note that each of these is an entirely separate service; they do not share records, user accounts, or other information. The Review E-Link is intended for testing your code, or for new yet-to-be-released features and interface functionality, and does not constitute release of STI information to the production OSTI.gov web site.

It is not required to maintain separate python library dependencies for these different environments; the library instance may connect to either service depending on how you instantiate it.

Note that documentation via python help functionality is provided for most of the library functions, via help(Elink) or help(Elink.post_new_record) for example.

from elinkapi import Elink

api = Elink(target="TARGET_URL", token="__Your_API_Token__")

Again, if you do not specify the TARGET_URL above, the production E-Link will be utilized. Specify https://review.osti.gov/elink2api/ as the TARGET_URL above to access the Review E-Link environment. Adjust the token value to the appropriate API user token for the correct target environment prior to utilizing the functions. The production environment endpoint value is https://www.osti.gov/elink2api/.

If you wish to interactively change either value, there are convenience methods available:

# change the TARGET to review
api.set_target_url("https://review.osti.gov/elink2api/")
# but also don't forget to change your token!
api.set_api_token("MYNEWTOKENVALUE")
# you may also view these settings in python
api.target
'https://review.osti.gov/elink2api/'
api.token
'MYNEWTOKENVALUE'

Note: Ensure site_ownership_code is a value to which your user account token has sufficient access to create records.

from elinkapi import Elink, Record, exceptions

api = Elink(token="__Your_API_Token__")

# Record with minimal fields to save
my_record_json = {
        "title": "A Dissertation Title",
        "site_ownership_code": "AAAA",
        "product_type": "TD"
        }
# Convert json to Record object
my_record = Record(**my_record_json)

saved_record = None
try:
    saved_record = api.post_new_record(my_record, "save")
except exceptions.BadRequestException as ve:
    # ve.message = "Site Code AAAA is not valid."
    # ve.errors provides more details:
    # [{"status":"400", "detail":"Site Code AAAA is not valid.", "source":{"pointer":"site_ownership_code"}}]

from elinkapi import Elink, Record, BadRequestException

# Record missing fields, will give 2 validation errors, one for 
# each missing field: title and product_type
my_invalid_record_json = {
    "site_ownership_code": "AAAA"
}

try:
    # The pydantic model will raise exceptions for the 2 missing 
    # fields - title and product_type
    my_record = Record(**my_invalid_record_json)
except Exception as e:
    print('Exception on Record creation')
    # pydantic will return "missing" required fields as below:
    # 2 validation errors for Record
    # product_type
    #    Field required [type=missing, input_value={'site_ownership_code': 'BBBB'}, input_type=dict]
    #    For further information visit https://errors.pydantic.dev/2.6/v/missing
    # title
    #    Field required [type=missing, input_value={'site_ownership_code': 'BBBB'}, input_type=dict]
    #    For further information visit https://errors.pydantic.dev/2.6/v/missing

my_invalid_record_json = {
    "title": "A Sample Title",
    "product_type": "TD",
    "site_ownership_code": "AAAA"
}

my_record = Record(**my_invalid_record_json)

saved_record = None
try:
    # The API will now return an error code on this call
    # because "AAAA" is not a valid site_ownership_code
    saved_record = api.post_new_record(my_record, "save")
except exceptions.BadRequestException as ve:
    # E-Link BadRequestException provides details of the API response:
    # ve.message = "Site Code AAAA is not valid."
    # ve.errors provides more details:
    # [{"status":"400", "detail":"Site Code AAAA is not valid.", "source":{"pointer":"site_ownership_code"}}]

When records are submitted to the E-Link API (either via POST for a new record, or PUT to update a revision) and validation is successful, the E-Link release processing services immediately begin to process the record. This often will transition the workflow_status of the current revision through the initial "SO" through "SV" to a final "R" of a successful release.

During this processing, a record might stop at the "SV" state (if awaiting media processing, for example) or end up in "SF" (submitted, but failed validations) state. In these cases, or if additional information is desired for fully-released records, the auditing logs are available to query, showing the processing states and workers involved, or any issues in processing that might be preventing release of the revision.

Simply use the get_single_record API function call with a given OSTI ID:

from elinkapi import Elink

api = Elink(token = "__Your_API_Token__")

record = api.get_single_record(osti_id_value)

# "raw" audit logs
record.audit_logs
[AuditLog(messages=['Revision passed validation'], status='SUCCESS', type='VALIDATOR', audit_date=datetime.datetime(2024, 2, 13, 17, 0, 4, 76000, tzinfo=TzInfo(UTC)))]

# obtain audit log information in JSON
[l.model_dump_json() for l in record.audit_logs]

['{"messages":["Revision passed validation"], "status":"SUCCESS", "type": "VALIDATOR", "audit_date":"2024-02-13T17:00:04.076000Z"}']

Audit logs will typically include references from "VALIDATOR", "RELEASER", "DOI", and other microservices as applicable. Each audit event will detail the date-time performed, and a status of either "SUCCESS" or "FAIL". This will provide more insight into the processing of individual submitted records. Once a revision fully releases (workflow_status="R") it should become available on the OSTI.gov search service (in production API instances).

Note that most API responses, including records and many of their component child data (identifiers, etc.) are returned as pydantic data classes. If information in JSON is desired, use the model_dump_json() method available to such output classes.

from elinkapi import Elink

api = Elink(token="__Your_API_Token__")

osti_id = 99999999

revision_history = None
try:
    revision_history = api.get_all_revisions(osti_id)
except Exception as e:
    # Handle the exception as needed

most_recent_revision = revision_history[0]
oldest_revision = revision_history[-1]

Various components of record history are available in each revision summary, such as the date_valid_start and date_valid_end, workflow_status, etc. The dates indicate the time frame during which that revision was "current"; if date_valid_end is null, this indicates that particular revision is the current one.

from elinkapi import Elink, NotFoundException, ForbiddenException

api = Elink(token = "__Your_API_Token__")

try:
    record = api.get_revision_by_number(4525922,2)

    record.json()

    print ("Revision valid from {} to {}".format(record.date_valid_start,
                                                 record.date_valid_end if record.date_valid_end else "Present"))
except NotFoundException:
    print ("Revision not on file.")
except ForbiddenException:
    print ("Permission to view record denied.")

from elinkapi import Elink

api = Elink(token = '__Your_API_Token__')

osti_id = 9999999
path_to_my_media = "/home/path/to/media.pdf"

saved_media = None
try:
    saved_media = api.post_media(osti_id, path_to_my_media)
except Exception as e:
    # Handle the exception as needed

from elinkapi import Elink

api = Elink(token = "___Your-API-Token___")

try:
    response = api.delete_record(99999, "Resource no longer available")
except Exception as e:
    # Handle any exceptions

from elinkapi import Elink

api = Elink(token = "___Your-API-Token___")

osti_id = 9999999
media_id = 71
reason = "Uploaded the wrong file"

response = None
try:
    response = api.delete_single_media(osti_id, media_id, reason)
except Exception as e:
    # Handle the exception as needed

from elinkapi import Elink

api = Elink(token = "___Your-API-Token___")

osti_id = 2300069
revision_id_left = 1
revision_id_right = 2

response = None
try:
    response = elinkapi.compare_two_revisions(osti_id, revision_id_left, revision_id_right)
except Exception as e:
    # Handle the exception as needed

from elinkapi import Elink, Query

api  = Elink(token = "___Your-API-Token___")

query = api.query_records(title = "science", product_type = "JA")

# see number of results
print (f"Query matched {query.total_rows} records")

# paginate through ALL results using iterator
for record in query:
    print (f"OSTI ID: {record.osti_id} Title: {record.title}")

Searches are limited via keywords specified in the query_records method call. Search term fields and further information is available in the online API documentation.

The following methods may alter parameters on existing Elink instances to alter or set values.

from elinkapi import Elink

# you may set these directly or alter them later
# note target defaults to "https://www.osti.gov/elink2api/" for the E-Link API
# to access Review or Beta environment, you may specify the target:
api = Elink(token = 'TOKENVALUE', target="https://review.osti.gov/elink2api/")

# change them
api.set_api_token("NEWTOKEN")
# change to PRODUCTION E-Link API
api.set_target_url("https://www.osti.gov/elink2api/")

Method:

set_api_token(api_token)

Returns: None

Params:

• api_token - str: Unique to user API token that can be generated from your E-Link Account page

Method:

set_target_url(url="https://www.osti.gov/elink2api"):

Returns: None

Params:

• url - str: The url to which all other module methods will direct their requests (default: {"https://www.osti.gov/elink2api"})

Method:

get_single_record(osti_id)

Returns: Record

Params:

• osti_id - int: ID that uniquely identifies an E-Link 2.0 Record

Method:

query_records(params)

Example:

api.query_records(title="science")

Returns: Query object

Params:

• params - dict: See here for the list of allowed query parameters.

Method:

reserve_doi(record)

Returns: Record

Params:

• record - Record: Metadata record that you wish to save to E-Link 2.0

Method:

post_new_record(record, state="save")

Returns: Record

Params:

• record - Record: Metadata record that you wish to send ("save" or "submit") to E-Link 2.0
• state - str: The desired submission state of the record ("save" or "submit") (default: {"save"})

Method:

update_record(osti_id, record, state="save")

Returns: Record

Params:

• osti_id - int: ID that uniquely identifies an E-Link 2.0 Record
• record - Record: Metadata record that you wish to make the new revision of OSTI ID
• state - str: The desired submission state of the record ("save" or "submit") (default: {"save"})

Method:

get_revision_by_number(osti_id, revision_number)

Returns: Record

Params:

• osti_id - int: ID that uniquely identifies an E-Link 2.0 Record
• revision_number - int: The specific revision number to retrieve (original record is 1 and each revision increments upward by 1)

Method:

get_revision_by_date(osti_id, date)

Returns: Record

Params:

• osti_id - int: ID that uniquely identifies an E-Link 2.0 Record
• date - datetime: Date on which you wish to search for a revision of a Record

Method:

get_all_revisions(osti_id)

Returns: RevisionHistory

Params:

• osti_id - int: ID that uniquely identifies an E-Link 2.0 Record

Method:

compare_two_revisions(osti_id, left, right)

Returns: List[RevisionComparison]

Params:

• osti_id - int: ID that uniquely identifies an E-Link 2.0 Record
• left - int: The first revision number to retrieve and compare to the right
• right - int The second revision number to retrieve and compare to the left

Method:

get_media(osti_id)

Returns: MediaInfo

Params:

• osti_id - int: ID that uniquely identifies an E-Link 2.0 Record

Method:

get_media_content(media_file_id)

Returns: Byte string of the media file content

Params:

• media_file_id - int: ID that uniquely identifies a media file associated with an E-Link 2.0 Record

Method:

post_media(osti_id, file_path, params=None, stream=None)

Returns: MediaInfo

Params:

• osti_id - int: ID that uniquely identifies an E-Link 2.0 Record
• file_path - str: Path to the media file that will be attached to the Record
• params - dict: "title" that can be associated with the media file "url" that points to media if not sending file (default: {None})
• stream - bool: Whether to stream the media file data, which has better performance for larger files (default: {False})

Method:

put_media(osti_id, media_id, file_path, params=None, stream=None)

Returns: MediaInfo

Params:

• osti_id - int: ID that uniquely identifies an E-Link 2.0 Record
• media_id - int: ID that uniquely identifies a media file associated with an E-Link 2.0 Record
• file_path - str: Path to the media file that will replace media_id Media
• params - dict: "title" that can be associated with the media file "url" that points to media if not sending file (default: {None})
• stream - bool: Whether to stream the media file data, which has better performance for larger files (default: {False})

Method:

delete_all_media(osti_id, reason)

Returns: True on success, False on failure

Params:

• osti_id - int: ID that uniquely identifies an E-Link 2.0 Record
• reason - str: reason for deleting all media

Method:

delete_single_media(osti_id, media_id, reason)

Returns: True on success, False on failure

Params:

• osti_id - int: ID that uniquely identifies an E-Link 2.0 Record
• media_id - int: ID that uniquely identifies a media file associated with an E-Link 2.0 Record
• reason - str: reason for deleting media

Each class is a pydantic model that validates the metadata's data types and enumerated values on instantiation of the class. Each may be imported directly:

from elinkapi import Record, Organization, Person, Query, Identifier, RelatedIdentifier, Geolocation, MediaInfo, MediaFile
from elinkapi import Revision, RevisionComparison

Matches the Metadata model described in E-Link 2.0's API documentation

Produced by API query searches, enables pagination and access to total count of rows matching the query. Query is iterable, and may use Python constructs to paginate all results as desired. Iterating over the Query returns metadata records individually.

See Searching and pagination for example and more information.

Provides:

• total_rows - int: Total count of records matching the query
• has_next() - boolean: True if there are more results to be fetched
• has_previous() - boolean: True if there is a previous page of results

Matches the Organizations model described in E-Link 2.0's API documentation

Matches the Persons model described in E-Link 2.0's API documentation

Matches the Identifiers model described in E-Link 2.0's API documentation

Matches the Related Identifiers model described in E-Link 2.0's API documentation

Information for any geolocation information (as pairs of latitude and longitude points) associated with a record's information.

Schema

Geolocation: {
    "type": str
    "label": str
    "points": List[Point]
}
Point: {
    "latitude": float
    "longitude": float
}

Example

{
    "type": "BOX",
    "label": "Utah FORGE",
    "points": [
        {
            "latitude": 38.5148,
            "longitude": -112.879748
        },
        {
            "latitude": 38.483935,
            "longitude": 112.916367
        }
    ]
}

Information for a "media set" associated with a record; that is, one or more media files, such as PDF or Word documents, or off-site URLs (for dataset records), along with any derived or processed files produced by media processing. Such derived files are usually cached URL content, text extracted from full text files, or OCR-processed PDF files as applicable.

Each media set is uniquely identified by the generated "media_id" value, and each "osti_id" record may be associated with one or more such media sets. The set contains individual "files" (see "Media File" below) making up its content.

Schema

[
    {
        "media_id": int,
        "revision": int,
        "access_limitations": List[str],
        "osti_id": int,
        "status": str,
        "added_by": int,
        "document_page_count": int,
        "mime_type": str,
        "media_title": str,
        "media_location": str,
        "media_source": str,
        "date_added": datetime,
        "date_updated": datetime,
        "date_valid_start": datetime,
        "date_valid_end": datetime,
        "files": List[MediaFile]
    }
]

Example

[
    {
        "media_id": 233743,
        "revision": 3,
        "access_limitations": [],
        "osti_id": 99238,
        "status": "P",
        "added_by": 34582,
        "document_page_count": 23,
        "mime_type": "application/pdf",
        "media_title": "PDF of technical report content",
        "media_location": "L",
        "media_source": "MEDIA_API_UPLOAD",
        "date_added": "1992-03-08T11:23:44.123+00:00",
        "date_updated": "2009-11-05T08:33:12.231+00:00",
        "date_valid_start": "2021-02-13T16:32:23.234+00:00",
        "date_valid_end": "2021-02-15T12:32:11.332+00:00",
        "files": []
    }
]

An individual media file within a "media set"; usually a PDF or Word document uploaded by the user, or a processed file derived from that upload. The "media_type" of "O" (for original) indicates the single user-provided document, which is usually the fulltext reference on OSTI.gov for this record. Derived files, such as text extracted (media_type="T"), cached URL content (media_type="C") will also be present in the media set.

Each media file is uniquely identified by its "media_file_id" sequence number, generated by OSTI when the files are uploaded or processed.

Schema

{
    "media_file_id": int,
    "media_id": int,
    "revision": int,
    "status": str,
    "media_type": str,
    "url_type": str,
    "added_by_user_id": int,
    "file_size_bytes": int,
    "date_file_added": datetime,
    "date_file_updated": datetime"
}

Example

{
    "media_file_id": 12001019,
    "media_id": 1900094,
    "revision": 2,
    "status": "ADDED",
    "media_type": "O",
    "url_type": "L",
    "added_by_user_id": 112293,
    "file_size_bytes": 159921,
    "date_file_added": "2023-12-20T22:13:16.668+00:00",
    "date_file_updated": "2023-12-20T22:13:16.668+00:00"
}

Schema

{
    "date_valid_start": datetime,
    "date_valid_end": datetime,
    "osti_id": int,
    "revision": int,
    "workflow_status": str
}

Example

{
    "date_valid_start": "2022-12-04T13:22:45.092+00:00",
    "date_valid_end": "2023-12-04T13:22:45.092+00:00",
    "osti_id": 2302081,
    "revision": 2,
    "workflow_status": "R"
}

Schema

[
    {
        "date_valid_start": datetime,
        "date_valid_end": datetime,
        "osti_id": int,
        "revision": int,
        "workflow_status": str
    }
]

Example

[
    {
        "pointer": "/edit_reason",
        "left": "API record creation",
        "right": "API metadata Update"
    },
    {
        "pointer": "/description",
        "left": "A custom description. Search on 'Allo-ballo holla olah'.",
        "right": "A NEW custom description. Search on 'Allo-ballo holla olah'."
    }
]

A number of enumerations are provided for use in Record construction or as field value vocabulary as applicable. The following list indicates the descriptions and uses of each of the provided value sets and where they apply. Each may be individually imported as needed:

from elinkapi import AccessLimitation, ProductType

>>> AccessLimitation.UNL
<AccessLimitation.UNL: 'Unlimited'>
>>> AccessLimitation.UNL.name
'UNL'
>>> help(AccessLimitation)
Help on class AccessLimitation in module elinkapi.record:

class AccessLimitation(enum.Enum)
 |  AccessLimitation(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)
 |  
 |  Access limitations, or distribution limitations, describe the intended audience
 |  or any restrictions to be placed upon the distribution of this product's metadata
 |  and/or related full text.  This may range from UNL (Unlimited), being essentially
 |  no restriction, to various levels of limited audience notations as desired.
 |  
 |  Note that many combinations are disallowed as conflicting; UNL may not be 
 |  combined with most others, for example, while OUO is often combined with various
 |  other levels (e.g., PROT, SSI, etc.)
 :

Defines one or more access or distribution limitations for the indicated Record, indicating various sensitivity markings or other details about how this product should be distributed via OSTI.gov or other OSTI output products, and its dissemination once fully processed. Note that a number of these (indicated in descriptions below) are legacy or historical in context, and generally will not be accepted as new product or revision data.

Most access limitations beyond UNL will require additional information for definitions in other required fields. Note that only certain combinations of access limitations are valid; sensitive limitations such as PROT, OUO, and the like may not be combined with UNL, for example.

CUI access limitation must be specified alone, and additional CUI markings designated in the access_limitation_other field.

For these enumerations, the Name is also the code value in the Record access_limitations array.

Describes the particular type of this journal publication, and is only applicable to ProductType JA.

Specific values for DOE-PAMS related data products only. Details publication status of the PAMS work.

Specific values for DOE-PAMS related patent data only. Provides state information of the patent.

Specific values for DOE-PAMS records only. Further defines the type of PAMS data record.

Indicates the type of product represented by this record. Each product type may require or disallow certain fields; for example, JA requires a number of "journal-related" fields, while most other types disallow information in these fields.

Current processing state of a given revision/record. Users generally submit in Saved ("SA"), SubmitReleasing ("SR"), or SubmitOSTI ("SO"); for the latter state, automated workflow processes will move through the other states ultimately to Released ("R"). Revisions will remain in Validated ("SV") state until for example media is attached and processed if applicable. Failure states, such as FailedValidation ("SF") and FailedRelease ("SX") should have explanations in the Record audit log information.

Various exceptions are raised via API calls, and may be imported and caught in the code. Using

from elinkapi import exceptions

will provide access to the various exception types for handling.

Generally raised when no API token value is provided when accessing E-Link.

Raised when attempting to query records, post new content to a site, or create/update records to which the API token has no permission.

Raised when provided query parameters or values are not valid or not understood, or if validation errors occurred during submission of metadata. Additional details are available via the errors list, each element containing the following information about the various validation issues:

• detail: an error message indicating the issue
• source: contains a "pointer" to the JSON tag element in error

Example:

[{
  "detail":"Site Code BBBB is not valid.",
  "source":{
    "pointer":"site_ownership_code"
  }}]

Raised when OSTI ID or requested resource is not on file.

Raised when attempting to attach duplicate media or URL to a given OSTI ID metadata.

Raised if E-Link back end services or databases have encountered an unrecoverable error during processing.