Publishing

Publish types

There are multiple types of publishing, which corresponds with item life cycle:

  • publish

  • correct

  • kill

  • unpublish

  • takedown

For each there is specific resource and service:

class apps.publish.content.publish.ArchivePublishService
class apps.publish.content.correct.CorrectPublishService
class apps.publish.content.kill.KillPublishService
class apps.publish.content.unpublish.UnpublishService
class apps.publish.content.take_down.TakeDownPublishService

all inheriting from base publish service

class apps.publish.content.common.BasePublishService

These in general handle validation and update item metadata.

Main steps

Publishing flow in Superdesk mainly consists of the next stages:

Small diagram showing a publishing flow

Note

In sections below ArchivePublishService will be used as an example reference.

Validation

When publishing starts, it first validates the item based on its content profile definition or in case content profile is missing it will get validators from db. There are different validators for different content types (text, package, picture, etc) and publish type.

apps.publish.content.publish.ArchivePublishService._validate()

Note

apps.validate.validate.ValidateService() is used for item validation

After the item is validated, associated items are validated to ensure that none of them are locked, killed, spiked, or recalled.

apps.publish.content.publish.ArchivePublishService._validate_associated_items()

Items in packages are also validated if were not published before. Package is considered not valid if any of its item is not valid.

apps.publish.content.publish.ArchivePublishService._validate_package()

Schema definition

When using content profiles or validators, you specify a schema for each field like:

"headline": {
    "type": "string",
    "required": true,
    "maxlength": 140,
    "minlength": 10
}

More info about validation rules in Eve docs.

Item metadata update

When item is valid, it gets some metadata updates:

  • firstpublished is set to publish_schedule datetime if scheduled or utcnow

  • operation is set to “publish”. Operation depends on publish types.
    This value defines which enqueue service will be used to enqueue an item.

    Enqueue services:

    enqueue_services = {
    ITEM_PUBLISH: EnqueuePublishedService(),
    ITEM_CORRECT: EnqueueCorrectedService(),
    ITEM_KILL: EnqueueKilledService(),
    ITEM_TAKEDOWN: EnqueueKilledService(published_state=CONTENT_STATE.RECALLED),
    ITEM_UNPUBLISH: EnqueueKilledService(published_state=CONTENT_STATE.UNPUBLISHED),
}

  • state is set based on action

  • _current_version is incremented

  • version_creator is set to current user

  • pubstatus is set to “usable”. Pubstatus depends on publish types.

  • expiry set item expiry

  • word_count update word count

apps.publish.content.publish.ArchivePublishService.on_update()

Note

If an item has associations, those are marked as used ArchivePublishService._mark_media_item_as_used()

Save item for enqueue

These changes are saved to archive collection and published collection.

Note

After item is saved to published collection,
apps.publish.enqueue.enqueue_published.apply_async() is executed immediately.
Celery beat also runs this task every 10 seconds

Client is notified that item is published via item:publish push notification. On client those items are not visible anymore in monitoring, only in desk output.

If there any updates to associated items and PUBLISH_ASSOCIATED_ITEMS is true then publish the associated items.

apps.publish.content.common.BasePublishService._publish_associated_items()

Processing

New items from published collection are further processed via async task:

which runs apps.publish.enqueue.EnqueueContent() command.

Note

It’s possible to run this command manually using:

python manage.py publish:enqueue

Enqueueing is done via:

All items with queue state: “pending” that are not scheduled or scheduled time has lapsed are quiried for processing.
item['operation'] which was set at item metadata update step, defines an enqueue service.
There are a lot of actions happen in EnqueueService:
  • get the subscribers:

    • get all active subscribers

    • filter the subscriber list based on the publish filter and global filters (if configured)

  • queue the content for subscribers EnqueueService.queue_transmission:

    • get formatter

    • format item

    • save result item into publish_queue

  • sends notification if no formatter has found for any of the formats configured in subscriber

  • publish item to content API if configured

Note

Rewrites are sent to subscribers that received the original item or the previous rewrite.

Output Formatters

Superdesk NINJS Schema in JSON.

Transmission

Last task is to send items to subscribers, that’s handled via another async task:

Note

It’s possible to start transmition manually:

python manage.py publish:transmit

This task runs every 10s.

Content Transmitters