Upgrade Swarm

The section describes how to upgrade a Swarm package, OVA, or tarball install to a newer release.

Tip

If you are not already running Swarm, none of these instructions apply to you. Instead, see the Swarm installation instructions.

Swarm upgrade process:

Upgrade a Swarm package installation

Important
  • Swarm runtime dependencies change between releases, you must check that your system satisfies the Swarm runtime dependencies before starting the upgrade, see Runtime dependencies.
  • Review the PHP requirements before you upgrade Swarm, see PHP.

  • Review the Helix server requirements before you upgrade Swarm, see Helix Core server requirements.
  • Helix server 2020.1 and later, permissions have changed for viewing and editing stream spec files in Swarm. To view and edit stream spec files in Swarm, the Swarm user must have admin permissions for the entire depot //...
Note
  • If you are upgrading from Swarm 2017.2 or earlier, run the Swarm index upgrade after you have validated your upgrade. This is the last step of the upgrade and ensures that the review activity history is displayed in the correct order on the Dashboard, and Reviews list pages.

  • If you are upgrading from Swarm 2020.2 or earlier and have userids that contain the forward slash (/) character, add AllowEncodedSlashes NoDecode to the VirtualHost block of your /etc/apache2/sites-enabled/perforce-swarm-site.conf file. For more information about the VirtualHost block, see Apache configuration.

Before you begin your Swarm upgrade

The Swarm Workflow feature was introduced in Swarm 2018.2 and was disabled by default, this feature is now enabled by default for Swarm 2019.2 and later.

If you are not currently using the Swarm workflow feature and you are using the strict and enforce triggers to control commits you have the following options:

  • Use the Swarm workflow feature: you must comment out your strict and enforce triggers and use the new workflow triggers.
  • Note

    Known limitations

    The workflow triggers do not support the following trigger functionality available in Swarm 2018.1 and earlier:

    • EXEMPT_FILE_COUNT
    • EXEMPT_EXTENSIONS

    To continue to use this trigger functionality, you must keep your existing enforce and strict triggers and disable the Workflow feature as shown below.

  • Continue to use the strict and enforce triggers: keep your existing enforce and strict triggers and Disable the Workflow feature. Support for these triggers will be dropped in a later release.

    Tip

    If you disable the workflow feature in the Swarm config.php file, workflow will not be processed by Swarm but a small overhead is still incurred by the Helix server each time it runs a workflow trigger script. This overhead can be eliminated by commenting out the swarm.enforce change-submit, swarm.strict change-content, and swarm.shelvesub shelve-submit workflow triggers.

Decide whether you want to use the workflow feature before you start your upgrade because this will determine which triggers you need to use. The trigger requirements are described in more detail in the Update your triggers stage of the Swarm upgrade.

Upgrade Swarm

Important

For the Swarm 2015.2 release, the packages have been renamed. The following instructions upgrade your Swarm packages to the latest versions.

Tip

From Swarm 2021.1, the Swarm package upgrade installs logrotate to manage your Swarm log rotation. If the package upgrade finds an existing custom logrotate configuration file for Swarm, the upgrade will notify you and give you details on how to disable the new logrotate configuration.

For information about the logrotate configuration, see Logrotate.

The following process attempts to minimize downtime, but a short period of downtime for Swarm users is unavoidable. There should be no downtime for your Helix server. After a successful upgrade, all Swarm users are logged out.

If you are using Swarm in a production environment, we encourage you to test this upgrade process in a non-production environment first.

  1. Follow the instructions for your OS distribution:
  2. Swarm generally has several major updates each year, and may occasionally have a patch update between major updates. To determine whether a Swarm update is available, follow the instructions for your OS distribution:
Tip

Swarm uses a Redis server to manage its caches. This is installed and configured on the Swarm machine during the upgrade. If you prefer to use your own Redis server, see Use your own Redis server.

Update your triggers

  1. Copy the new Swarm trigger script to your Helix Core server machine. The trigger script is SWARM_ROOT/p4-bin/scripts/swarm-trigger.pl, and requires installation of Perl 5.08+ (use the latest available) on the Helix server machine. If Swarm is using SSL, then the triggers also require the IO::Socket::SSL Perl module.

    Warning

    Do not overwrite any existing trigger script at this time. Give the script a new name, for example: swarm-trigger-new.pl.

  2. Configure the Swarm trigger script by creating, in the same directory on the Helix server machine, swarm-trigger.conf. It should contain:

    Note

    If you already have a swarm-trigger.conf file, no additional configuration is required.

    # SWARM_HOST (required)
    # Hostname of your Swarm instance, with leading "http://" or "https://".
    SWARM_HOST="http://my-swarm-host"
    
    # SWARM_TOKEN (required)
    # The token used when talking to Swarm to offer some security. To obtain the
    # value, log in to Swarm as a super user and select 'About Swarm' to see the
    # token value.
    SWARM_TOKEN="MY-UUID-STYLE-TOKEN"
    
    # ADMIN_USER (optional) Do not use if the Workflow feature is enabled (default)
    # For enforcing reviewed changes, optionally specify the normal Perforce user
    # with admin privileges (to read keys); if not set, will use whatever Perforce
    # user is set in environment.
    ADMIN_USER=
    
    # ADMIN_TICKET_FILE (optional) Do not use if the Workflow feature is enabled (default)
    # For enforcing reviewed changes, optionally specify the location of the
    # p4tickets file if different from the default ($HOME/.p4tickets).
    # Ensure this user is a member of a group with an 'unlimited' or very long
    # timeout; then, manually login as this user from the Perforce server machine to
    # set the ticket.
    ADMIN_TICKET_FILE=				
    										
    # VERIFY_SSL (optional)
    # If HTTPS is being used on the Swarm web server, then this controls whether
    # the SSL certificate is validated or not. By default this is set to 1, which
    # means any SSL certificates must be valid. If the web server is using a self
    # signed certificate, then this must be set to 0.
    # set the ticket.
    VERIFY_SSL=1

    Fill in the required SWARM_HOST and SWARM_TOKEN variables with the configuration from any previous Swarm trigger script, typically swarm-trigger.pl.

    Tip

    The ADMIN_USER and ADMIN_TICKET variables were used by the 'enforce triggers' in Swarm 2019.1 and earlier. They can be removed unless you are explicitly disabling workflow and using the deprecated 'enforce triggers'.

    Note

    Swarm 2015.4 and earlier: Swarm trigger script files were available as shell scripts in these earlier Swarm versions, typically swarm-trigger.sh.

    Swarm must now use a Perl trigger script file, typically swarm-trigger.pl.

  3. On Linux: ensure that the script is executable:

    $ sudo chmod +x swarm-trigger-new.pl

  4. Rename the new trigger script:

    On Linux:

    $ mv swarm-trigger-new.pl swarm-trigger.pl

    On Windows:

    C:\> ren swarm-trigger-new.pl swarm-trigger.pl

  5. Update the triggers in your Helix server.

    Warning
    • The swarm.shelvedel shelve-delete trigger line was added to Swarm in version 2018.1 and updated in version 2020.1.

      • Upgrading from Swarm 2017.4 and earlier: add the swarm.shelvedel shelve-delete trigger line to the Helix server trigger table if it is not already present, see Update the Helix server triggers table to run the trigger script.
      • Upgrading from Swarm 2018.x and 2019.x: replace the existing swarm.shelvedel shelve-delete trigger line in the Helix server trigger table with the one supplied in the Swarm version you are upgrading to.
    • Workflow feature:

      The Workflow feature is enabled by default in Swarm 2019.2 and later. The trigger lines required when workflow is enabled are different to those required when workflow is disabled:

    1. Run the Swarm trigger script to capture (using Ctrl+C on Windows and Linux, Command+C on Mac OSX) the trigger lines that should be included in the Perforce trigger table:

      On Linux:

      $ ./swarm-trigger.pl -o

      On Windows:

      C:\> path/to/perl swarm-trigger.pl -o

    2. As a Perforce user with super privileges, update the Perforce trigger table by running p4 triggers command and replacing any swarm.* lines with the previously captured trigger line output (using Ctrl+V on Windows and Linux, Command+V on Mac OSX).
    Important

    If you previously customized the Swarm trigger lines, perhaps to apply various Trigger options, be sure to repeat those customizations within the updated trigger lines.

Validate your upgrade

Tip

When Swarm starts it verifies the Redis cache, during this time you cannot log in to Swarm. The time taken to verify the Redis cache depends on the number of users, groups, and projects Swarm has. Start-up time can be improved by persisting the memory cache. You can persist the memory cache by disabling background saves and enabling append saves in the redis-server.conf file, see Redis server configuration file.

Check that your upgraded Swarm instance is working correctly by doing the following:

  1. Create a new changelist that:
    1. Contains at least one modified file
    2. Contains the #review keyword in the changelist description
  2. Right click on the new changelist in P4V and click Shelve Files...
  3. Important

    Do not select Request New Swarm Review... because this method uses the API and will not fully test the Swarm triggers.

  4. Check that a new Swarm review is created for the changelist.
    • If a Swarm review is created, the Swarm triggers are working.
    • If a Swarm review is not created, see below.
Note

If a new Swarm review is not created when you validate your upgrade, you may be missing some Swarm triggers on the Helix server. Periodically new triggers are added to Swarm and these need to be installed when you upgrade, see the Update your triggers section above. For more information about Swarm trigger configuration on your Helix server, see Helix Core server configuration for Swarm.

Swarm index upgrade

If you are upgrading from Swarm 2017.2 or earlier you should run the index upgrade, this ensures that the review activity history is displayed in the correct order on the Dashboard, and Reviews list pages.

Note

If you are upgrading from Swarm version 2017.3 or later, the index upgrade step is not required.

The index upgrade process can be configured to suit your Swarm system specifications. See Upgrade index for details.

Run the upgrade as an Admin user by visiting the following URL:

http://SWARM-HOST/upgrade
Note

After the Swarm upgrade, on the first visit to some Swarm pages, users might see a message to perform a browser hard refresh. This happens because we are updating the UI and content of some Swarm pages, so the user's page cache is no longer valid and requires a hard refresh to load the upgraded page. For example, for Chrome on Windows/Linux [CTRL]+[F5] and for Chrome on MacOS [Command]+[Shift]+[R].

All done!

Upgrade a Swarm tarball installation

Warning
  • Swarm runtime dependencies change between releases, you must check that your system satisfies the Swarm runtime dependencies before starting the upgrade, see Runtime dependencies.
  • P4PHPClosedThe PHP interface to the Helix API, which enables you to write PHP code that interacts with a Helix server machine. should be upgraded to the version included in the new Swarm release.
    • If you have already configured PHP to use the Swarm-provided P4PHP (as recommended), this happens automatically.
    • If you have manually installed P4PHP in some other fashion, configure Swarm to use the version of P4PHP included in the new Swarm tarball before you perform any of the upgrade steps below. See PHP configuration for details.

    Swarm package, OVA and tarball installations: 2 versions of P4PHP are supplied for each PHP 7 version supported by Swarm. They are located in the p4-bin/bin.linux26x86_64 directory.

    • perforce-php7x.so compatible with systems using SSL 1.0.2
    • perforce-php7x-ssl1.1.1.so compatible with systems using SSL 1.1.1 (by default Ubuntu 18.04 and Ubuntu 20.04 use SSL 1.1.1)

    Where x is the version of PHP 7.

    If the perforce.ini file is not pointing at the correct version of P4PHP and you connect to an SSL enabled Helix server:

    • The Swarm web-page will not load and you might see a Connection Reset error.
    • There might be an undefined symbol: SSLeay message in the Apache error log
  • Review the PHP requirements before you upgrade Swarm, see PHP.

  • Review the Helix server requirements before you upgrade Swarm, see Helix Core server requirements.
  • Helix server 2020.1 and later, permissions have changed for viewing and editing stream spec files in Swarm. To view and edit stream spec files in Swarm, the Swarm user must have admin permissions for the entire depot //...
Note

OVA upgrade process:

Note
  • If you are upgrading from Swarm 2017.2 or earlier, run the Swarm index upgrade after you have validated your upgrade. This is the last step of the upgrade and ensures that the review activity history is displayed in the correct order on the Dashboard, and Reviews list pages.

  • If you are upgrading from Swarm 2020.2 or earlier and have userids that contain the forward slash (/) character, add AllowEncodedSlashes NoDecode to the VirtualHost block of your /etc/apache2/sites-enabled/perforce-swarm-site.conf file. For more information about the VirtualHost block, see Apache configuration.

Before you begin your Swarm upgrade

The Swarm Workflow feature was introduced in Swarm 2018.2 and was disabled by default, this feature is now enabled by default for Swarm 2019.2 and later.

If you are not currently using the Swarm workflow feature and you are using the strict and enforce triggers to control commits you have the following options:

  • Use the Swarm workflow feature: you must comment out your strict and enforce triggers and use the new workflow triggers.
  • Note

    Known limitations

    The workflow triggers do not support the following trigger functionality available in Swarm 2018.1 and earlier:

    • EXEMPT_FILE_COUNT
    • EXEMPT_EXTENSIONS

    To continue to use this trigger functionality, you must keep your existing enforce and strict triggers and disable the Workflow feature as shown below.

  • Continue to use the strict and enforce triggers: keep your existing enforce and strict triggers and Disable the Workflow feature. Support for these triggers will be dropped in a later release.

    Tip

    If you disable the workflow feature in the Swarm config.php file, workflow will not be processed by Swarm but a small overhead is still incurred by the Helix server each time it runs a workflow trigger script. This overhead can be eliminated by commenting out the swarm.enforce change-submit, swarm.strict change-content, and swarm.shelvesub shelve-submit workflow triggers.

Decide whether you want to use the workflow feature before you start your upgrade because this will determine which triggers you need to use. The trigger requirements are described in more detail in the Update your triggers stage of the Swarm upgrade.

Upgrade Swarm

The following process attempts to minimize downtime, but a short period of downtime for Swarm users is unavoidable. There should be no downtime for your Helix server. After a successful upgrade, all Swarm users are logged out.

If you are using Swarm in a production environment, we encourage you to test this upgrade process in a non-production environment first.

CentOS/RHEL: PHP 7.x and Apache 2.4 installation

CentOS and RHEL do not have PHP 7.x and Apache 2.4 by default so you must upgrade your system before you can upgrade Swarm. This process is only required the first time you upgrade to PHP 7.x.

Configure Redis on the Swarm machine

Follow the instructions for the Swarm version you are upgrading from:

Now that the Swarm Redis service is configured for the Swarm machine, start your Swarm upgrade, see Upgrade Swarm.

Upgrade Swarm

The steps in this section describe how to upgrade Swarm using the provided archive file. SWARM_ROOT refers to the current Swarm installation.

Tip

For OVA installations, SWARM_ROOT is /opt/perforce/swarm.

  1. Download the new TAR file from https://www.perforce.com/downloads/helix-swarm.

  2. Expand the new swarm.tgz:

    $ tar -zxf swarm.tgz
    

    The contents of swarm.tgz are expanded into a top-level folder named swarm-version, where version corresponds to the version downloaded. This directory is identified as SWARM_NEW below.

  3. Move SWARM_NEW to be a peer of SWARM_ROOT:

    $ mv SWARM_NEW SWARM_ROOT/../
  4. Copy the SWARM_ROOT/data/config.php file from SWARM_ROOT to SWARM_NEW:

     $ cp -p SWARM_ROOT/data/config.php SWARM_NEW/data/
    
  5. Create the queue token directory:

     $  mkdir SWARM_NEW/data/queue
  6. Copy the existing trigger token(s):

     $ sudo cp -pR SWARM_ROOT/data/queue/tokens SWARM_NEW/data/queue/
  7. Assign correct ownership to the new Swarm's data directory:

    $ sudo chown -R www-data SWARM_NEW/data
    Note

    The www-data user above is an example of what the web server user name might be, and can vary based on distribution or customization. For example, the user is typically apache for Red Hat/Fedora/CentOS, www-data for Debian/Ubuntu, wwwrun for SuSE, _www for Mac OSX.

Update your triggers

  1. Copy the new Swarm trigger script to your Helix Core server machine. The trigger script is SWARM_ROOT/p4-bin/scripts/swarm-trigger.pl, and requires installation of Perl 5.08+ (use the latest available) on the Helix server machine. If Swarm is using SSL, then the triggers also require the IO::Socket::SSL Perl module.

    Warning

    Do not overwrite any existing trigger script at this time. Give the script a new name, for example: swarm-trigger-new.pl.

  2. Configure the Swarm trigger script by creating, in the same directory on the Helix server machine, swarm-trigger.conf. It should contain:

    Note

    If you already have a swarm-trigger.conf file, no additional configuration is required.

    # SWARM_HOST (required)
    # Hostname of your Swarm instance, with leading "http://" or "https://".
    SWARM_HOST="http://my-swarm-host"
    
    # SWARM_TOKEN (required)
    # The token used when talking to Swarm to offer some security. To obtain the
    # value, log in to Swarm as a super user and select 'About Swarm' to see the
    # token value.
    SWARM_TOKEN="MY-UUID-STYLE-TOKEN"
    
    # ADMIN_USER (optional) Do not use if the Workflow feature is enabled (default)
    # For enforcing reviewed changes, optionally specify the normal Perforce user
    # with admin privileges (to read keys); if not set, will use whatever Perforce
    # user is set in environment.
    ADMIN_USER=
    
    # ADMIN_TICKET_FILE (optional) Do not use if the Workflow feature is enabled (default)
    # For enforcing reviewed changes, optionally specify the location of the
    # p4tickets file if different from the default ($HOME/.p4tickets).
    # Ensure this user is a member of a group with an 'unlimited' or very long
    # timeout; then, manually login as this user from the Perforce server machine to
    # set the ticket.
    ADMIN_TICKET_FILE=				
    										
    # VERIFY_SSL (optional)
    # If HTTPS is being used on the Swarm web server, then this controls whether
    # the SSL certificate is validated or not. By default this is set to 1, which
    # means any SSL certificates must be valid. If the web server is using a self
    # signed certificate, then this must be set to 0.
    # set the ticket.
    VERIFY_SSL=1

    Fill in the required SWARM_HOST and SWARM_TOKEN variables with the configuration from any previous Swarm trigger script, typically swarm-trigger.pl.

    Tip

    The ADMIN_USER and ADMIN_TICKET variables were used by the 'enforce triggers' in Swarm 2019.1 and earlier. They can be removed unless you are explicitly disabling workflow and using the deprecated 'enforce triggers'.

    Note

    Swarm 2015.4 and earlier: Swarm trigger script files were available as shell scripts in these earlier Swarm versions, typically swarm-trigger.sh.

    Swarm must now use a Perl trigger script file, typically swarm-trigger.pl.

  3. On Linux: ensure that the script is executable:

    $ sudo chmod +x swarm-trigger-new.pl

  4. Rename the new trigger script:

    On Linux:

    $ mv swarm-trigger-new.pl swarm-trigger.pl

    On Windows:

    C:\> ren swarm-trigger-new.pl swarm-trigger.pl

  5. Update the triggers in your Helix server.

    Warning
    • The swarm.shelvedel shelve-delete trigger line was added to Swarm in version 2018.1 and updated in version 2020.1.

      • Upgrading from Swarm 2017.4 and earlier: add the swarm.shelvedel shelve-delete trigger line to the Helix server trigger table if it is not already present, see Update the Helix server triggers table to run the trigger script.
      • Upgrading from Swarm 2018.x and 2019.x: replace the existing swarm.shelvedel shelve-delete trigger line in the Helix server trigger table with the one supplied in the Swarm version you are upgrading to.
    • Workflow feature:

      The Workflow feature is enabled by default in Swarm 2019.2 and later. The trigger lines required when workflow is enabled are different to those required when workflow is disabled:

    1. Run the Swarm trigger script to capture (using Ctrl+C on Windows and Linux, Command+C on Mac OSX) the trigger lines that should be included in the Perforce trigger table:

      On Linux:

      $ ./swarm-trigger.pl -o

      On Windows:

      C:\> path/to/perl swarm-trigger.pl -o

    2. As a Perforce user with super privileges, update the Perforce trigger table by running p4 triggers command and replacing any swarm.* lines with the previously captured trigger line output (using Ctrl+V on Windows and Linux, Command+V on Mac OSX).
    Important

    If you previously customized the Swarm trigger lines, perhaps to apply various Trigger options, be sure to repeat those customizations within the updated trigger lines.

Replace your old Swarm instance with your new Swarm instance

Replace your old Swarm with the new Swarm. Downtime occurs in this step.

$ sudo apache2ctl stop; mv SWARM_ROOT SWARM.old; mv SWARM_NEW SWARM_ROOT; sudo apache2ctl start

Note

If you see the following error message when you start Swarm, Swarm is using the wrong version of P4PHP. The latest version of P4PHP is included in the Swarm tarball but you must configure Swarm to use that version of P4PHP. For instructions about how to configure Swarm to use the new version of P4PHP, see PHP configuration.

Image of the P4PHP version error message

Delete the Swarm config cache

Delete the Swarm config cache to force Swarm to use any new and updated modules in the upgrade. To delete the Swarm config cache, make the following curl request as an admin user:

curl -u "username:password" -X DELETE "https://myswarm.url/api/v10/cache/config/"

For more information on deleting the config cache, see Swarm config cache file delete.

Validate your upgrade

Tip

When Swarm starts it verifies the Redis cache, during this time you cannot log in to Swarm. The time taken to verify the Redis cache depends on the number of users, groups, and projects Swarm has. Start-up time can be improved by persisting the memory cache. You can persist the memory cache by disabling background saves and enabling append saves in the redis-server.conf file, see Redis server configuration file.

Check that your new Swarm instance is working correctly by doing the following:

  1. Create a new changelist that:
    1. Contains at least one modified file
    2. Contains the #review keyword in the changelist description
  2. Right click on the new changelist in P4V and click Shelve Files...
  3. Important

    Do not select Request New Swarm Review... because this method uses the API and will not fully test the Swarm triggers.

  4. Check that a new Swarm review is created for the changelist.
    • If a Swarm review is created, the Swarm triggers are working.
    • If a Swarm review is not created, see below.
Note

If a new Swarm review is not created when you validate your upgrade, you may be missing some Swarm triggers on the Helix server. Periodically new triggers are added to Swarm and these need to be installed when you upgrade, see the Update your triggers section above. For more information about Swarm trigger configuration on your Helix server, see Helix Core server configuration for Swarm.

Swarm index upgrade

If you are upgrading from Swarm 2017.2 or earlier you should run the index upgrade, this ensures that the review activity history is displayed in the correct order on the Dashboard, and Reviews list pages.

Note

If you are upgrading from Swarm version 2017.3 or later, the index upgrade step is not required.

The index upgrade process can be configured to suit your Swarm system specifications. See Upgrade index for details.

Run the upgrade as an Admin user by visiting the following URL:

http://SWARM-HOST/upgrade
Note

After the Swarm upgrade, on the first visit to some Swarm pages, users might see a message to perform a browser hard refresh. This happens because we are updating the UI and content of some Swarm pages, so the user's page cache is no longer valid and requires a hard refresh to load the upgraded page. For example, for Chrome on Windows/Linux [CTRL]+[F5] and for Chrome on MacOS [Command]+[Shift]+[R].

All done!