mysqldump for longer term storage, but in some cases the data set is large enough that mysqldump is prohibitive.
In that case, you can enable external replication using an extra permission.
This guide explains replication on the Upsun side only; you must also set up and maintain your own replica instance. Consult the MySQL or MariaDB documentation for details.
Create a replication user
To set up replication you need to create a replication-enabled user. For each database that you’d like to replicate, you need to assign areplication permission/role, under a corresponding endpoint:
The preceding example:
- Creates a
replicatoruser. - Grants read-only and table locking rights on the
maindatabase (namelySelect_priv,Show_view_priv,Create_tmp_table_priv,Lock_tables_privprivileges). - Grants global replication rights (namely
Repl_slave_privandRepl_client_privprivileges). - Grants flushing rights (
Reload_privused for flushing before reading the binary log position). - If at least one
replicationpermission is defined, the bin-logging is enabled on the primary server, which is essential for the replication.
Define a relationship for the new endpoint
Even if your application won’t access the replication endpoint, you still need to expose it to an application as a relationship so that you can connect to it over SSH. Add a new relationship to your application container:.upsun/config.yaml
Get the primary’s binary log co-ordinates
Open the MySQL CLI to thereplication relationship, either by accessing the credentials while on the app container or using the following command.
FLUSH TABLES WITH READ LOCK. Keep this session running - exiting it releases the lock. Get the current position in the binary log by running SHOW MASTER STATUS:
File and Position details. If binary logging has just been enabled, these are blank. Now, with the lock still in place, copy the data from the primary to the replica.
Login to the app container, then run:
mysqldump has completed, you can release the lock on the primary by running UNLOCK TABLES.
Set up the replica
Configure the replica
As mentioned above you have to set up a replica on your own. Assuming that you have a running MariaDB/MySQL replica instance, give the replica a uniqueserver_id (distinct from primary). You can find out primary’s server_id by running:
server_id number (e.g. server_id+1) in your replica config (e.g. my.cnf) under:
Set up SSH tunneling
You need to set up an SSH tunnel from the replica server to the primary, tunneled through the application.The SSH tunnel is interrupted every time the environment redeploys. For replication to continue you must set up an auto-restart for the tunnel. There are many ways to do so that are out of the scope of this documentation.
- Manually using SSH. Using this method makes it easier to set up an auto-restart for the tunnel. Consult the SSH documentation for details on setting up the tunnel and the auto-restart.
-
Run the following Upsun CLI command
(replacing
<VariableBlock name="BRANCH_NAME" />with the name of your production branch):This command opens local SSH tunnels to all services accessible from the application.
Binary log retention and cleanup
When replication is disabled, the MariaDB service stops managing the binary logs and they remain on the file system. You must delete these logs manually. If the remote replica has been unreachable for some time, these logs can consume a significant amount of storage.Start the Replica
Once the data has been imported, you are ready to start replicating. Begin by running aCHANGE MASTER TO, making sure that MASTER_LOG_FILE matches the file and MASTER_LOG_POS the position returned by the earlier SHOW MASTER STATUS on the Upsun database. For example:
<VariableBlock name="REPLICATION_HOST" /> varies depending on the SSH tunneling configuration you have, and the <VariableBlock name="REPLICATION_PASSWORD" /> can be obtained by running upsun ssh env.
Now start the replica with the START SLAVE command:
SHOW SLAVE STATUS command:
Slave_IO_Running and Slave_SQL_Running should be Yes:
[Optional/Troubleshooting] Skipping invalid binary log queries
In some cases, after applying primary’s dump to the replica and starting the replica, you might experience replication errors (Slave_SQL_Running: No and Error: in the output of SHOW SLAVE STATUS \G above). Each of such errors needs a careful inspection, but you might be able to just skip some of them. For example:
SQL_SLAVE_SKIP_COUNTER (which corresponds to skipping the next N events from the primary) to a larger value.