yetangitu/funkwhale
1
0
Fork 0
mirror of https://code.eliotberriot.com/funkwhale/funkwhale.git synced 2025-10-03 09:49:23 +02:00
Code Issues Releases Wiki Activity

docs/admin: update music importing documentation

Browse source 
It still referenced pre-1.0 functionality
This commit is contained in:
xeruf 2021-10-09 21:06:59 +02:00
parent 9d42e4ea84
commit 98c6c60b6e
1 changed files with 67 additions and 66 deletions
Download patch file Download diff file Expand all files Collapse all files

133
docs/admin/importing-music.rst
View file

@ -1,21 +1,21 @@
Importing music from the server
===============================
Funkwhale can import music files that are located on the server assuming
they readable by the Funkwhale application. Your music files should contain at
least an ``artist``, ``album`` and ``title`` tags, but we recommend you tag
it extensively using a proper tool, such as Beets or Musicbrainz Picard.
Funkwhale can import music files saved on the server
assuming they are readable by the `funkwhale` user.
Your music files should contain at least
``artist``, ``album`` and ``title`` tags,
but we recommend you tag extensively using a proper tool,
such as Beets or Musicbrainz Picard.
Funkwhale supports two different import modes:
- copy (the default): files are copied into Funkwhale's internal storage. This means importing a 1GB library will result in the same amount of space being used by Funkwhale.
- :ref:`in-place <in-place-import>` (when the ``--in-place`` is provided): files are referenced in Funkwhale's DB but not copied or touched in anyway. This is useful if you have a huge library, or one that is updated by an external tool such as Beets.
- copy(only before 1.0): files are copied into Funkwhale's internal storage. This means importing a 1GB library will result in the same amount of space being used by Funkwhale.
- :ref:`in-place <in-place-import>` (default): files are referenced in Funkwhale's DB but not copied or touched in anyway. This is useful if you have a huge library, or one that is updated by an external tool such as Beets.
.. note::
In Funkwhale 1.0, **the default behaviour will change to in-place import**
Regardless of the mode you're choosing, import works as described below, assuming your files are located in
Regardless of the mode you're choosing,
import works as described below,
assuming your files are located in
``/srv/funkwhale/data/music``:
.. code-block:: bash
@ -24,17 +24,18 @@ Regardless of the mode you're choosing, import works as described below, assumin
python api/manage.py import_files $LIBRARY_ID "/srv/funkwhale/data/music/" --recursive --noinput
.. note::
You'll have to create a library in the Web UI before to get your library ID. Simply visit
https://yourdomain/content/libraries/ to create one.
You have to create a library in the Web UI to get your library ID.
Simply visit https://yourdomain/content/libraries/ to create one.
Library IDs are available in library urls or sharing link. In this example:
Library IDs are part of the library url or sharing link.
For example, the library ID of
https://funkwhale.instance/content/libraries/769a2ae3-eb3d-4aff-9f94-2c4d80d5c2d1,
the library ID is 769a2bc3-eb1d-4aff-9f84-2c4d80d5c2d1
is 769a2bc3-eb1d-4aff-9f84-2c4d80d5c2d1
You can use only the first characters of the ID when calling the command, like that:
``export LIBRARY_ID="769a2bc3"``
When you use docker, the ``/srv/funkwhale/data/music`` is mounted from the host
When you use docker, ``/srv/funkwhale/data/music`` is mounted from the host
to the ``/music`` directory on the container:
.. code-block:: bash
@ -50,25 +51,24 @@ When you installed Funkwhale via ansible, you need to call a script instead of P
/srv/funkwhale/manage import_files $LIBRARY_ID "/srv/funkwhale/data/music/" --recursive --noinput
The import command supports several options, and you can check the help to
get details::
The import command supports several options,
check the help for details::
docker-compose run --rm api python manage.py import_files --help
.. note::
For the best results, we recommend tagging your music collection through
`Picard <http://picard.musicbrainz.org/>`_ in order to have the best quality metadata.
We recommend tagging your music collection using `Picard <http://picard.musicbrainz.org/>`_ to have the best quality metadata.
.. note::
This command is idempotent, meaning you can run it multiple times on the same
files and already imported files will simply be skipped.
This command is idempotent,
meaning you can run it multiple times on the same files
and already imported files are simply skipped.
.. note::
At the moment, only Flac, OGG/Vorbis and MP3 or AIFF files with ID3 tags are supported
At the moment, only Flac, OGG/Vorbis and MP3 or AIFF files with ID3 tags are supported.
.. _in-place-import:
@ -76,49 +76,42 @@ get details::
In-place import
^^^^^^^^^^^^^^^
By default, the CLI-importer will copy imported files to Funkwhale's internal
storage. This means importing a 1GB library will result in the same amount
of space being used by Funkwhale.
While this behaviour has some benefits (easier backups and configuration),
it's not always the best choice, especially if you have a huge library
to import and don't want to double your disk usage.
The CLI importer supports an additional ``--in-place`` option that triggers the
following behaviour during import:
1. Imported files are not store in Funkwhale anymore
2. Instead, Funkwhale will store the file path and use it to serve the music
Because those files are not managed by Funkwhale, we offer additional
configuration options to ensure the webserver can serve them properly:
Because imported files are not managed by Funkwhale,
we offer additional configuration options
to ensure the webserver can serve them properly:
- :data:`MUSIC_DIRECTORY_PATH <config.settings.common.MUSIC_DIRECTORY_PATH>`
- :data:`MUSIC_DIRECTORY_SERVING_PATH <config.settings.common.MUSIC_DIRECTORY_SERVE_PATH>`
We recommend you symlink all your music directories into ``/srv/funkwhale/data/music``
and run the `import_files` command from that directory. This will make it possible
to use multiple music directories, without any additional configuration
on the webserver side.
and run the `import_files` command from that directory.
This will make it possible to use multiple music directories,
without any additional configuration on the webserver side.
For instance, if you have a NFS share with your music mounted at ``/media/mynfsshare``,
For instance, if you have an NFS share
with your music mounted at ``/media/mynfsshare``,
you can create a symlink like this::
ln -s /media/mynfsshare /srv/funkwhale/data/music/nfsshare
And import music from this share with this command::
And import music from the share::
export LIBRARY_ID="<your_libary_id>"
python api/manage.py import_files $LIBRARY_ID "/srv/funkwhale/data/music/nfsshare/" --recursive --noinput --in-place
On docker setups, it will require a bit more work, because while the ``/srv/funkwhale/data/music`` is mounted
in containers, symlinked directories are not.
Docker setups require a bit more work,
because while the ``/srv/funkwhale/data/music`` is mounted in containers,
symlinked directories are not.
To fix that, you can use bind mounts instead of symbolic links, as it replicates the source directory tree. With the previous NFS share, it would go this way::
To fix that, you can use bind mounts instead of symbolic links,
as they replicate the source directory tree.
With the previous NFS share, use this command::
mount --bind /media/mynfsshare /srv/funkwhale/data/music/nfsshare
If you want to go with symlinks, ensure each symlinked directory is mounted as a volume as well in your ``docker-compose.yml`` file::
If you want to go with symlinks,
ensure each symlinked directory is mounted as a volume
as well as in your ``docker-compose.yml`` file::
celeryworker:
volumes:
@ -137,9 +130,10 @@ If you want to go with symlinks, ensure each symlinked directory is mounted as a
Metadata updates
^^^^^^^^^^^^^^^^
When doing an import with in ``in-place`` mode, the importer will also check and update existing entries
found in the database. For instance, if a file was imported, the ID3 Title tag was updated, and you rerun a scan,
Funkwhale will pick up the new title. The following fields can be updated this way:
When doing an import with in ``in-place`` mode,
the importer will also check and update existing entries found in the database.
For instance, if the ID3 Title tag of an existing song was updated since the last scan, Funkwhale picks up the new title.
The following fields can be updated this way:
- Track mbid
- Track title
@ -159,13 +153,16 @@ Funkwhale will pick up the new title. The following fields can be updated this w
React to filesystem events with ``--watch``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If you have a really big library or one that is updated quite often, running the ``import_files`` command by hand
may not be practical. To help with this use case, the ``import_files`` command supports a ``--watch`` flag that will observes filesystem events
instead of performing a full import.
If you have a really big library or update it regularly,
running the ``import_files`` command by hand may not be practical.
For this use case,
the ``import_files`` command supports a ``--watch`` flag
through which it observes filesystem events instead of performing a full import.
File creation, move, update and removal are handled when ``--watch`` is provided:
File creation, move, update and removal
are handled when ``--watch`` is provided:
- Files created in the watched directory are imported immediatly
- Files created in the watched directory are imported immediately
- If using ``in-place`` mode, files updates trigger a metadata update on the corresponding entries
- If using ``in-place`` mode, files that are moved and known by Funkwhale will see their path updated in Funkwhale's DB
- If using ``in-place`` mode, files that are removed and known by Funkwhale will be removed from Funkwhale's DB
@ -173,7 +170,8 @@ File creation, move, update and removal are handled when ``--watch`` is provided
Pruning dangling metadata with ``--prune``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Funkwhale is, by design, conservative with music metadata in its database. If you remove a file from Funkwhale's DB,
Funkwhale is, by design, conservative with music metadata in its database.
If you remove a file from Funkwhale's DB,
the corresponding artist, album and track object won't be deleted by default.
If you want to prune dangling metadata from the database once the ``import_files`` command is over, simply add the ``--prune`` flag.
@ -182,18 +180,20 @@ This also works in with ``--watch``.
Album covers
^^^^^^^^^^^^
Whenever possible, Funkwhale will import album cover, with the following precedence:
Whenever possible, Funkwhale obtains album covers for tracks,
with the following precedence:
1. It will use the cover embedded in the audio files themeselves, if any (Flac/MP3 only)
2. It will use a cover.jpg or a cover.png file from the imported track directory, if any
3. It will fetch cover art from musicbrainz, assuming the file is tagged correctly
1. The cover embedded in the audio files themeselves, if any (Flac/MP3 only)
2. Use a cover.jpg or a cover.png file from the imported track directory, if any
3. Fetch cover art from musicbrainz, assuming the file is tagged correctly
Getting demo tracks
^^^^^^^^^^^^^^^^^^^
If you do not have any music on your server but still want to test the import
process, you can call the following methods do download a few albums licenced
under creative commons (courtesy of Jamendo):
If you do not have any music on your server
but want to test the import process,
you can call the following methods
to download a few albums licenced under creative commons (courtesy of Jamendo):
.. parsed-literal::
@ -202,4 +202,5 @@ under creative commons (courtesy of Jamendo):
chmod +x download-tracks.sh
./download-tracks.sh music.txt
This will download a bunch of zip archives (one per album) under the ``data/music`` directory and unzip their content.
This will download a bunch of zip archives (one per album)
under the ``data/music`` directory and unzip their content.