Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error current heartbeatlag (9223372036.85s) is too high #1444

Open
ritxman opened this issue Aug 19, 2024 · 1 comment
Open

Error current heartbeatlag (9223372036.85s) is too high #1444

ritxman opened this issue Aug 19, 2024 · 1 comment

Comments

@ritxman
Copy link

ritxman commented Aug 19, 2024

Hi all, I want to ask about gh-ost migration.
When I run gh-ost migration on my virtual machine and the progress reached 100%, it got error log current heartbeatlag 9223372036.85s is too high, it needs to be less than both --max-lag-millis (1.50s) and --cut-over-lock-timeout-seconds (3.00) to continue.
Here is the current specification:

  1. gh-ost version is 1.1.6
  2. using MariaDB version 11.4

What I have done are:

  1. Upgrading the gh-ost version from 1.1.5 into 1.1.6
  2. Running the gh-ost command without --max-lag-millis
  3. Reseting the master database to refresh the replica
  4. Run ANALYZE TABLE to refresh the performance schema
  5. Also I have set the timezone setting into UTC+7 to comply with my current timezone in Indonesia by running this SQL: SET GLOBAL time_zone = '+07:00'; and SET time_zone = '+07:00';

So I'm curious about the printed value of TimeSinceLastHeartbeatOnChangelog(), I decide to debug it on my local by print its value, but it shows 0001-01-01 00:00:00 +0000 UTC, hence the heartbeat lag value become extremely huge.

Do you have any idea why this happens? Will appreciate any help given for this issue.
Thanks in advance!

Attachments:
Screen Shot 2024-08-19 at 12 53 59

@omkarkulkarni2704
Copy link

Ensure performance_schema is enabled and set --heartbeat-interval 1s in your gh-ost command to fix the heartbeat lag issue.

  • Ensure performance_schema is configured correctly:
SET GLOBAL performance_schema = ON;
  • Add the --heartbeat-interval option in your gh-ost command:
--heartbeat-interval 1s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants