Superdesk CLI¶
In superdesk/server
folder you can find manage.py
file which you can use
to run commands from CLI:
$ python manage.py
With no other argument it will output list of all available commands. To run specific command you use its name and any params needed:
$ python manage.py users:create -u admin -p admin
Creating new command¶
You can create new commands extending superdesk.Command
and registering it using
superdesk.command()
call:
import superdesk
class HelloWorldCommand(superdesk.Command):
option_list = [
superdesk.Option('--name', '-n', required=True),
]
def run(self, name):
print('hello {}'.format(name))
superdesk.command('hello:word', MyCommand())
We use Flask-Script under the hood so you can get more info there.
Superdesk commands¶
app:clean_images
¶
- class CleanImages¶
This command will remove all the images from the system which are not referenced by content.
It checks the media type and calls the correspoinding function as s3 and mongo requires different approaches for handling multiple files. Probably running
db.repairDatabase()
is needed in Mongo to shring the DB size.Example:
$ python manage.py app:clean_images
app:deleteArchivedDocument
¶
- class DeleteArchivedDocumentCommand¶
Deletes a Text Archive document(s) from both Mongodb and ElasticSearch.
It deletes digital package and the story given by id. It accepts one or more ids separated by space.
Example:
$ manage.py app:deleteArchivedDocument 588c1df11d41c80928015601 588c1b901d41c805dce70df0
app:index_from_mongo
¶
- class IndexFromMongo¶
Index the specified mongo collection in the specified elastic collection/type.
This will use the default APP mongo DB to read the data and the default Elastic APP index.
Use
-f all
to index all collections.Example:
$ python manage.py app:index_from_mongo --from=archive $ python manage.py app:index_from_mongo --all
app:initialize_data
¶
app:flush_elastic_index
¶
- class FlushElasticIndex¶
Flush elastic index.
It removes elastic index, creates a new one and index it from mongo. You must specify at least one elastic index to flush:
--sd
(superdesk) or--capi
(content api)Example:
$ python manage.py app:flush_elastic_index --sd $ python manage.py app:flush_elastic_index --capi $ python manage.py app:flush_elastic_index --sd --capi
app:prepopulate
¶
app:populate
¶
app:rebuild_elastic_index
¶
- class RebuildElasticIndex¶
Rebuild the elastic indexes from existing data.
It creates new index with same alias as the configured index, puts the new mapping and removes the old index.
Example:
$ python manage.py app:rebuild_elastic_index $ python manage.py app:rebuild_elastic_index --resource=items $ python manage.py app:rebuild_elastic_index --resource=archive
app:run_macro
¶
- class RunMacro¶
Executes a macro by given name and optional keyword arguments.
Example:
$ app:run_macro --name clean_keywords --kwargs {"repo":"archived"}
app:scaffold_data
¶
app:updateArchivedDocument
¶
- class UpdateArchivedDocumentCommand¶
Update metadata for a document in both Mongodb and ElasticSearch by supplying ids field to update and the value for the update
- --field
Field name
- --value
Value to be set in the field
- --parseNeeded
Optional. True if value is a complex type, not an int or string
Example:
$ manage.py app:updateArchivedDocument --ids='["588c1df11d41c80928015601","588c1b901d41c805dce70df0"]' --field=anpa_category --value=[{"scheme" : null,"qcode" : "f","subject" : "04000000","name" : "Finance"}] --parseNeeded=True
archive:remove_expired
¶
audit:purge
¶
- class PurgeAudit¶
Purge audit collection.
Entries are purged if the related item is no longer in ‘Production’. Other entries are deleted after the configured time period.
Example:
$ python manage.py audit:purge $ python manage.py audit:purge --expiry_minutes=20
content_api:remove_expired
¶
- class RemoveExpiredItems¶
Remove expired items from the content_api items collection.
By default no items expire there, you can change it using
CONTENT_API_EXPIRY_DAYS
config.Example:
$ python manage.py content_api:remove_expired
data:generate_update
¶
- class GenerateUpdate¶
Generate a file where to define a new data update.
Example:
$ python manage.py data:generate_update --resource=archive
data:upgrade
¶
- class Upgrade¶
Runs all the new data updates available.
If
data_update_id
is given, runs new data updates until the given one.Example:
$ python manage.py data:upgrade
data:downgrade
¶
- class Downgrade¶
Runs the latest data update backward.
If
data_update_id
is given, runs all the data updates backward until the given one.Example:
$ python manage.py data:downgrade
ingest:clean_expired
¶
ingest:provider
¶
- class AddProvider¶
Add ingest provider.
Example:
$ python manage.py ingest:provider --provider='{"update_schedule" : { "minutes" : 5, "seconds" : 0 }, "idle_time" : { "hours" : 0, "minutes" : 0 }, "content_expiry" : 2880, "name" : "aap-demo", "source" : "aap-demo", "feeding_service" : "rss", "config" : { "url" : "https://abcnews.go.com/abcnews/primetimeheadlines", "field_aliases" : [ ] }, "feed_parser" : null, "content_types" : [ "text" ]}'
ingest:update
¶
legal_archive:import
¶
legal_publish_queue:import
¶
publish:enqueue
¶
publish:transmit
¶
- class PublishContent¶
Deliver items from
publish_queue to destinations.
Example:
$ python manage.py publish:transmit
session:gc
¶
- class RemoveExpiredSessions¶
Remove expired sessions from db.
Using
SESSION_EXPIRY_MINUTES
config.Example:
$ python manage.py session:gc
schema:migrate
¶
- class SchemaMigrateCommand¶
Migrate elastic schema if needed, should be triggered on every deploy.
It compares version set in code (latest) to one stored in db and only updates schema if those are different.
Current version is read from settings and fallbacks to superdesk.SCHEMA_VERSION, so that you can avoid migration via settings file if needed.
Example:
$ python manage.py schema:migrate
Storage Dumps and Records¶
Note
You can check Database Recording for details on how to use following
storage:…
commands.
- class StorageDump(func=None)¶
Dump collections from MongoDB
Note: this command should only be used for development purpose. Use MongoDB official commands to dump or restore databases in production.
Dump whole collection into either separate JSON files (one per collection) or a single one (with a single root object).
Single file are easier to copy, while separate JSON files are easier to check.
A dump is full Superdesk save, as opposed to a record which only store change made in a database (more like a diff).
Example:
Do a full database dump with a name and description to a single file:
$ python manage.py storage:dump -n "demo-instance" -D "this dump includes some test data (desks, users) to run a basic demo of Superdesk" --single
- class StorageRestore(func=None)¶
Restore MongoDB collections dumped with
storage:dump
Example:
$ storage:restore foobar_superdesk_dump
- class StorageStartRecording(func=None)¶
Record changes made in database until the command is stopped
This command is intended for developers to help producing specific state (e.g. for tests), or to create a specific Superdesk instance e.g. for a demo.
If you specify a base dump (with
--base-dump
), the database will be restored to this dump before starting the record, and it will be associated with the record.If no base dump is specified, the database should be in the same state as when the record has been done when you restore it.
If you use the
--full-document
option, the whole document will be stored in the record in case of update (instead of just diff), this will result in a bigger dump file, but may be applied to a database even if it was not exactly in the same state as when the record has been done.You may want to use
--collection
to record change only on the collections you’re interested in, and avoir side effects.Example:
Record change in vocabularies only, with a name and description, and base on “base_test_e2e_dump” dump:
$ python manage.py storage:record -b "base_test_e2e_dump" -c vocabularies -n "test_categories"-D "prepare instance for categories end-to-end tests"
- class StorageRestoreRecord(func=None)¶
Restore Superdesk record
This command is to be used with a record dump, not a full database archive.
Example:
$ storage:restore-record record-for-some-e2e-test
- class StorageList(func=None)¶
List Superdesk Dumps and Records
- class StorageMigrateDumps(func=None)¶
Apply migration scripts on all dumps and records
Note: the backend MUST NOT be running while this command is used.
Migration scripts are first applied on each records: changes on collections used in the record are merged to the record.
Then for full dump, migration are normally applied and dumps are updated.
The whole process may be long.
Be sure to validate and if necessary correct results before committing anything (specially for records).
Example:
$ python manage.py storage:upgrade-dumps
- class RemoveExportedFiles¶
Remove files from storage that were used for exporting items
Example:
$ python manage.py storage:remove_exported $ python manage.py storage:remove_exported --expire-hours=12
- class CreateUserCommand¶
Create a user with given username, password and email.
If user with given username exists it’s noop.
Example:
$ python manage.py users:create -u admin -p admin -e 'admin@example.com' --admin
- class ImportUsersCommand¶
Imports users from JSON or CSV file.
The file is a list of users, where the fields can be:
username
(String), mandatory if there is already a user with given username, it will be skipped.email
(String), mandatory if there is already a user with given email, it will be skipped.password
(String)first_name
(String)last_name
(String)sign_off
(String)role
(String): name of an existing role to assign. The role must exist, otherwise the user will be skipped. Note that the role is case sensitive.
For JSON, it’s a list of object, each user must map field name to value.
For CSV, fields names can be put in the first row of the CSV file. If the first row doesn’t contain fields names, mapping between columns and fields must be done using
-f
(fields mut be specified in the same order as columns).File extension is used to detect if the file is JSON or CSV.
Examples:
$ python manage.py users:import path/to/users_list.json $ python manage.py users:import -f username -f email -f first_name -f last_name path/to/users_list.csv
or, if the first row of your CSV file contains the fields names:
$ python manage.py users:import path/to/users_list.csv
- class ImportUserProfileFromADCommand¶
Responsible for importing a user profile from Active Directory (AD) to Mongo.
This command runs on assumption that the user executing this command and the user whose profile need to be imported need not to be the same. Uses ad_username and ad_password to bind to AD and then searches for a user identified by username_to_import and if found imports into Mongo.
Example:
$ python manage.py users:copyfromad --ad_username=ad_uname --ad_password=123 --username_to_import=admin
- class GetAuthTokenCommand¶
Gets auth token.
Generate an authorization token to be able to authenticate against the REST api without starting the client the copy the authorization header.
Example:
$ python manage.py users:get_auth_token --username=admin --password=123123
- class HashUserPasswordsCommand¶
Hash all the user passwords which are not hashed yet.
Example:
$ python manage.py users:hash_passwords
- class GenerateVocabularies¶
Generate vocabularies.json from flat file
This command generate the vocabularies.json file from a flat text file having the following structure:
Vocabulary Label: item 1 item 2 item 3
Vocabulary label must end by a colon (:). Indendation is not significant. By default vocabularies maps to an id with the label in lower case where spaces have been replaced by underscode (_). It is possible though to map to an other label/id. For this, you can specify a json file using the
--key-map
option. It expects a path to json file mapping from vocabulary label set in the flat text file to superdesk id (which will also be used as label). e.g.:{ "author types": "author roles", "news statuses": null, "news urgencies": "urgency", "genres": { "name": "Genre", "extra": {"selection_type": "multi selection"}, }, "": { "extra": {"type": "unmanageable"} } }
plain text name will be used as label/id.
null means that the value must be ignored
a dict helps to manage more complicated mapping. See below for the keys which can be used.
empty string (“”) can be used as key when extra must be applied to all non ignored vocabularies
Key which can be used in dictionary:
key
explanation
name
used as label and _id (in lower case and with spaces replaced by underscodes)
extra
a dictionary which will be merged to default one. Useful to add missing keys/values.
You can also specify a path to a file containing a json array using
--base
which will contain the base vocabularies to use. If not specified, an empty array will be used.The generated json will be written to
vocabularies.json
in the current directory.
- class UpdateVocabulariesInItemsCommand¶
Update documents in archive and published collections which contain CV related fields: subject, genre, place, anpa_category with corresponding data from vocabularies.
Example:
$ python manage.py vocabularies:update_archive
- class ImportCommand¶
Import articles into archives.
Example:
$ python manage.py xml:import ninjs data/sample.json
- class RegisterClient¶
Register a client to authentication server
A client name is needed, and an id and password will be generated and displayed once the client is registered.
The client will be allowed to access the given scopes using the
--scope
argument (this argument may be used several times).Example: Register a new client with name
my Superdesk client
and allowed to reach items in archive and get users:$ python manage.py auth_server:register_client "my Superdesk client" -s ARCHIVE_READ -s USERS_READ
- class UpdateClient¶
Update an existing client
You need to specify the client ID to update. Specify the parameters that you want to update. You can use
--password
to set a new password, keep empty to generate a new password.Examples: Change name of client with id ````:
$ python manage.py auth_server:update_client 0102030405060708090a0b0c –name “some new name”
Regenerate password of client with id
5dad7ee94269dd1d5a78e6a1
:$ python manage.py auth_server:update_client 5dad7ee94269dd1d5a78e6a1 –password
Change scope of client with id
5dad7f064269dd1d5a78e6a2
to allow only ARCHIVE_READ:$ python manage.py auth_server:update_client 5dad7f064269dd1d5a78e6a2 -s ARCHIVE_READ
- class UnregisterClient¶
Remove a previously registered client Example:
Unregister client with id
0102030405060708090a0b0c
:$ python manage.py auth_server:unregister_client 0102030405060708090a0b0c
- class ListClients¶
List clients registered with auth server
The client will be listed with their scopes.
Example:
$ python manage.py auth_server:list_clients