Skip to content

MySql /Mariadb error: max key length is 767 byte #4894

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
2 of 7 tasks
ulorentz opened this issue Nov 26, 2017 · 31 comments
Closed
2 of 7 tasks

MySql /Mariadb error: max key length is 767 byte #4894

ulorentz opened this issue Nov 26, 2017 · 31 comments
Labels
💊 bug Something isn't working status: needs feedback Tell me more about it

Comments

@ulorentz
Copy link

  • Gogs version (or commit ref): 0.11.34
  • Git version: 2.11.0
  • Operating system: Debian 9
  • Database (use [x]):
    • PostgreSQL
    • MySQL
    • MSSQL
    • SQLite
  • Can you reproduce the bug at https://try.gogs.io:
    • Yes (provide example URL)
    • No
    • Not relevant
  • Log gist (usually found in log/gogs.log):

Description

I tried to install from binary, I created the database on MariaDB, I launched ./gogs web and opened the installing page on browser. I inserted all needed but I received the error "Database setting is not correct: Error 1709: Index column size too large. The maximum column size is 767 bytes."
I think it's a problem related to MariaDB char set size, but I wasn't able to solve it.
...

@playmyskay
Copy link

I have the same error.
It seems to be a similar problem which is also described here:
https://discuss.gogs.io/t/solved-mysql-error-1064-while-running-first-install/1604/3

I tried these options in mysql (mariadb):
SET GLOBAL innodb_file_format=Barracuda;
SET GLOBAL innodb_file_per_table=ON;
innodb_large_prefix=1

But the create statements of the tables should have (which is not the case):
ROW_FORMAT=DYNAMIC;

@ulorentz
Copy link
Author

For me, it doesn't work. Do you confirm those commands don't solve the issue?

@ImAnOwl
Copy link

ImAnOwl commented Nov 27, 2017

have the same problem, but I have already read, upgrading mysql to 5.7 should solve the problem. I'll write later again, when i solved it

@ulorentz
Copy link
Author

Debian 9 uses MariaDB 10.1, that should be mySQL compatible, but it's not mySQL.

@ImAnOwl
Copy link

ImAnOwl commented Nov 27, 2017

I upgraded to MariaDB 10.2 and now it works. :)

@playmyskay
Copy link

playmyskay commented Nov 27, 2017

Here a short description how i get it working.. it is very laborious but it works for me:

  1. Create your gogs database as usual
  2. Visit the install page
  3. Type in your data (database, user, etc.)
  4. Click on Button "Install"
    -> It will fail with: 'Error 1709: Index column size too large. The maximum column size is 767 bytes.'
    BUT the tables were partially created.

To see which tables are currently present:
use gogs; (gogs = your gogs database name!)
SHOW TABLE STATUS;

  1. Alter the tables with the following command:
    ALTER TABLE _tablename_ ROW_FORMAT=DYNAMIC;

  2. Now go back to step 2 and repeat it. Always alter all "new" tables which were created.

After 4 or 5 rounds you get every needed table and it's done.

I am not completely sure whether you have to set the above mentioned options. I have done that.

Good luck.

@ulorentz
Copy link
Author

It works! Thank you very much.
It would be nice to have this process automatic (what should change in gogs code?).

@intersel
Copy link

@playmyskay Thanks a lot for the solution that worked for me too on
mysql Ver 15.1 Distrib 10.0.32-MariaDB, for debian-linux-gnu (i686) using readline 5.2
on debian 8

@m2nlight
Copy link
Contributor

m2nlight commented Dec 4, 2017

Because of mysql 5.6 (includes prior versions) InnoDB max index length is 767 bytes, mysql 5.7.7 is up to 3072 bytes.
If some varchar column's length is 255, when the character format is utf-8 needs 255*3=765 bytes for index length, It's OK.
But, an utf8mb needs 255*4=1020 bytes for index length.

Solutions:

  • UPGRADE the mysql to 5.7.7(Mariadb 10.2.2)
  • Change the utf8mb column length to 191 (191*4=764)
  • Change the utf8mb to utf8
  • set innodb_file_format=Barracuda, innodb_large_prefix=on and create table using ROW_FORMAT=DYNAMIC or COMPRESSED (default for 5.7.7)

I will fix it later.

@laurivosandi
Copy link

Hi, on Debian 9 with MariaDB this sill seems to be much of an hassle. Just to save someone else a bit time, dump this repeatedly to the SQL prompt until Gogs install finishes:

ALTER TABLE `access` ROW_FORMAT=DYNAMIC;
ALTER TABLE `access_token` ROW_FORMAT=DYNAMIC;
ALTER TABLE `action` ROW_FORMAT=DYNAMIC;
ALTER TABLE `attachment` ROW_FORMAT=DYNAMIC;
ALTER TABLE `collaboration` ROW_FORMAT=DYNAMIC;
ALTER TABLE `comment` ROW_FORMAT=DYNAMIC;
ALTER TABLE `deploy_key` ROW_FORMAT=DYNAMIC;
ALTER TABLE `follow` ROW_FORMAT=DYNAMIC;
ALTER TABLE `hook_task` ROW_FORMAT=DYNAMIC;
ALTER TABLE `issue` ROW_FORMAT=DYNAMIC;
ALTER TABLE `issue_label` ROW_FORMAT=DYNAMIC;
ALTER TABLE `issue_user` ROW_FORMAT=DYNAMIC;
ALTER TABLE `label` ROW_FORMAT=DYNAMIC;
ALTER TABLE `login_source` ROW_FORMAT=DYNAMIC;
ALTER TABLE `milestone` ROW_FORMAT=DYNAMIC;
ALTER TABLE `mirror` ROW_FORMAT=DYNAMIC;
ALTER TABLE `protect_branch` ROW_FORMAT=DYNAMIC;
ALTER TABLE `protect_branch_whitelist` ROW_FORMAT=DYNAMIC;
ALTER TABLE `public_key` ROW_FORMAT=DYNAMIC;
ALTER TABLE `pull_request` ROW_FORMAT=DYNAMIC;
ALTER TABLE `release` ROW_FORMAT=DYNAMIC;
ALTER TABLE `repository` ROW_FORMAT=DYNAMIC;
ALTER TABLE `star` ROW_FORMAT=DYNAMIC;
ALTER TABLE `two_factor` ROW_FORMAT=DYNAMIC;
ALTER TABLE `two_factor_recovery_code` ROW_FORMAT=DYNAMIC;
ALTER TABLE `upload` ROW_FORMAT=DYNAMIC;
ALTER TABLE `user` ROW_FORMAT=DYNAMIC;
ALTER TABLE `watch` ROW_FORMAT=DYNAMIC;
ALTER TABLE `webhook` ROW_FORMAT=DYNAMIC;

@ghost
Copy link

ghost commented Jan 4, 2018

worked for me

ALTER TABLE `access` ROW_FORMAT=DYNAMIC;
ALTER TABLE `access_token` ROW_FORMAT=DYNAMIC;
ALTER TABLE `action` ROW_FORMAT=DYNAMIC;
ALTER TABLE `attachment` ROW_FORMAT=DYNAMIC;
ALTER TABLE `collaboration` ROW_FORMAT=DYNAMIC;
ALTER TABLE `comment` ROW_FORMAT=DYNAMIC;
ALTER TABLE `deploy_key` ROW_FORMAT=DYNAMIC;
ALTER TABLE `email_address` ROW_FORMAT=DYNAMIC;
ALTER TABLE `follow` ROW_FORMAT=DYNAMIC;
ALTER TABLE `hook_task` ROW_FORMAT=DYNAMIC;
ALTER TABLE `issue` ROW_FORMAT=DYNAMIC;
ALTER TABLE `issue_label` ROW_FORMAT=DYNAMIC;
ALTER TABLE `issue_user` ROW_FORMAT=DYNAMIC;
ALTER TABLE `label` ROW_FORMAT=DYNAMIC;
ALTER TABLE `login_source` ROW_FORMAT=DYNAMIC;
ALTER TABLE `milestone` ROW_FORMAT=DYNAMIC;
ALTER TABLE `mirror` ROW_FORMAT=DYNAMIC;
ALTER TABLE `notice` ROW_FORMAT=DYNAMIC;
ALTER TABLE `org_user` ROW_FORMAT=DYNAMIC;
ALTER TABLE `protect_branch` ROW_FORMAT=DYNAMIC;
ALTER TABLE `protect_branch_whitelist` ROW_FORMAT=DYNAMIC;
ALTER TABLE `public_key` ROW_FORMAT=DYNAMIC;
ALTER TABLE `pull_request` ROW_FORMAT=DYNAMIC;
ALTER TABLE `release` ROW_FORMAT=DYNAMIC;
ALTER TABLE `repository` ROW_FORMAT=DYNAMIC;
ALTER TABLE `star` ROW_FORMAT=DYNAMIC;
ALTER TABLE `team` ROW_FORMAT=DYNAMIC;
ALTER TABLE `team_repo` ROW_FORMAT=DYNAMIC;
ALTER TABLE `team_user` ROW_FORMAT=DYNAMIC;
ALTER TABLE `two_factor` ROW_FORMAT=DYNAMIC;
ALTER TABLE `two_factor_recovery_code` ROW_FORMAT=DYNAMIC;
ALTER TABLE `upload` ROW_FORMAT=DYNAMIC;
ALTER TABLE `user` ROW_FORMAT=DYNAMIC;
ALTER TABLE `watch` ROW_FORMAT=DYNAMIC;
ALTER TABLE `webhook` ROW_FORMAT=DYNAMIC;

@sxl5513
Copy link

sxl5513 commented Jan 11, 2018

I use mysql5.6 , this error exist, your mysql version is 5.7??

@gabriellima
Copy link

Just to add that, for me, I had to also alter database and table's character set.

The full list of commands was:

ALTER TABLE `access` ROW_FORMAT=DYNAMIC;
ALTER TABLE `access_token` ROW_FORMAT=DYNAMIC;
ALTER TABLE `action` ROW_FORMAT=DYNAMIC;
ALTER TABLE `attachment` ROW_FORMAT=DYNAMIC;
ALTER TABLE `collaboration` ROW_FORMAT=DYNAMIC;
ALTER TABLE `comment` ROW_FORMAT=DYNAMIC;
ALTER TABLE `deploy_key` ROW_FORMAT=DYNAMIC;
ALTER TABLE `follow` ROW_FORMAT=DYNAMIC;
ALTER TABLE `hook_task` ROW_FORMAT=DYNAMIC;
ALTER TABLE `issue` ROW_FORMAT=DYNAMIC;
ALTER TABLE `issue_label` ROW_FORMAT=DYNAMIC;
ALTER TABLE `issue_user` ROW_FORMAT=DYNAMIC;
ALTER TABLE `label` ROW_FORMAT=DYNAMIC;
ALTER TABLE `login_source` ROW_FORMAT=DYNAMIC;
ALTER TABLE `milestone` ROW_FORMAT=DYNAMIC;
ALTER TABLE `mirror` ROW_FORMAT=DYNAMIC;
ALTER TABLE `protect_branch` ROW_FORMAT=DYNAMIC;
ALTER TABLE `protect_branch_whitelist` ROW_FORMAT=DYNAMIC;
ALTER TABLE `public_key` ROW_FORMAT=DYNAMIC;
ALTER TABLE `pull_request` ROW_FORMAT=DYNAMIC;
ALTER TABLE `release` ROW_FORMAT=DYNAMIC;
ALTER TABLE `repository` ROW_FORMAT=DYNAMIC;
ALTER TABLE `star` ROW_FORMAT=DYNAMIC;
ALTER TABLE `two_factor` ROW_FORMAT=DYNAMIC;
ALTER TABLE `two_factor_recovery_code` ROW_FORMAT=DYNAMIC;
ALTER TABLE `upload` ROW_FORMAT=DYNAMIC;
ALTER TABLE `user` ROW_FORMAT=DYNAMIC;
ALTER TABLE `watch` ROW_FORMAT=DYNAMIC;
ALTER TABLE `webhook` ROW_FORMAT=DYNAMIC;

ALTER DATABASE `gogs` CHARACTER SET utf8 COLLATE utf8_general_ci;

ALTER TABLE `access` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `access_token` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `action` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `attachment` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `collaboration` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `comment` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `deploy_key` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `email_address` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `follow` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `hook_task` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `issue` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `issue_label` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `issue_user` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `label` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `login_source` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `milestone` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `mirror` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `notice` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `org_user` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `protect_branch` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `protect_branch_whitelist` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `public_key` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `pull_request` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `release` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `repository` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `star` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `team` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `team_repo` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `team_user` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `two_factor` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `two_factor_recovery_code` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `upload` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `user` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `watch` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE `webhook` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;

@zenny
Copy link

zenny commented Feb 3, 2018

Didn't work! :-( Details at: #4891 (comment)

MariaDB [gogs]> show table status;
+------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+--------------------+----------+----------------+---------+
| Name | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time         | Update_time | Check_time | Collation          | Checksum | Create_options | Comment |
+------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+--------------------+----------+----------------+---------+
| user | InnoDB |      10 | Compact    |    0 |              0 |       16384 |               0 |            0 |         0 |              1 | 2018-02-03 23:57:08 | NULL        | NULL       | utf8mb4_general_ci |     NULL |                |         |
+------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+--------------------+----------+----------------+---------+
1 row in set (0.00 sec)

MariaDB [gogs]> ALTER TABLE _user_ ROW_FORMAT=DYNAMIC;
ERROR 1146 (42S02): Table 'gogs._user_' doesn't exist

@m2nlight
Copy link
Contributor

m2nlight commented Feb 4, 2018

The easiest way:
Upgrade mariadb version to 10.2 or higher.
Upgrade mysql version to 5.7 or higher.

@bluems bluems mentioned this issue Feb 5, 2018
7 tasks
@DJManas
Copy link

DJManas commented Feb 5, 2018

I would love to upgrade to 10.2, but since it is not arm ready yet, I have a problem :-(

EDIT: No problem, converted tables collation also and it works on 10.1 as well. Thanks

@YakovL
Copy link

YakovL commented Mar 7, 2018

I've passed through this issue, so let me summarize:

  1. the method by playmyskay worked for me
    • I set both encoding and row_format; to speed up the process, I used the list shared by gabriellima and used its 2 parts (except for the ALTER DATABASE line) each time (after each run of .gogs web) instead of putting specific lines (for specific tables)
    • I've also used the ALTER DATABASE line suggested by gabriellima in the beginning of the process
  2. I'm on MariaDB 10.1 (strange, I've installed it a couple of days ago and it's not latest 10.2.x), I haven't tried m2nlight's suggestion but I'd recomment those who just started to deal with this issue to try this first
  3. there's also a patch by m2nlight which I haven't tried and which is probably another straightforward solution
    • fixing this in production is discussed here

@unknwon unknwon added 💊 bug Something isn't working status: await It is time consuming labels Mar 8, 2018
@unknwon unknwon added the status: needs feedback Tell me more about it label Mar 9, 2018
@unknwon unknwon removed the status: await It is time consuming label Mar 9, 2018
@unknwon unknwon added this to the 0.12 milestone Mar 9, 2018
@unknwon
Copy link
Member

unknwon commented Mar 9, 2018

This is claimed to be fixed by merging #4913, please help test on develop branch.

@ghost
Copy link

ghost commented May 21, 2018

Problem still exists, nothing above fixes it or works, please re-open and raise importance level thank you

EDIT:
I have fixed the install, instead of copy/paste the SQL (which failed for me) I created a databse in virtualmin using the "additional options" to match the character set found in the file here https://github.com/gogits/gogs/blob/master/scripts/mysql.sql

CREATE DATABASE IF NOT EXISTS gogs CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;

So I created a new database in virtualmin, before clicking create you click "additional options" and then you select the utf8mb4 character set along with the utf8mb4_general_ci collation

Now Gogs installs a new version

Problem is it does not detect my repositories inside of /gogs-repositories/ from my previous installation. Can anyone show me how to import my repositories from that directory? I cannot seem to find a "backup repos" or "import/restore repos" section.

Thank you

@m2nlight
Copy link
Contributor

About backup and restore gogs datas.

Backup:
$ ./gogs backup

Restore:
$ ./gogs restore --from gogs-backup-[timestamp].zip

Restore SSH and Repo-hooks:

  1. Administrator account login gogs
  2. Goto "Admin Panel" - "Operations"
  3. Click "Rewrite '.ssh/authorized-keys' file"
  4. Click "Resync pre-receive, update and post-receive hooks of all repositories"

@unknwon
Copy link
Member

unknwon commented May 22, 2018

@MyS5Droid upgrade MySQL to 5.7 as per request of https://gogs.io/docs/installation.

@ghost
Copy link

ghost commented May 22, 2018

@m2nlight Do you know how to restore it from the directory though? I did not know there was CLI command for backup/restore with an archive file :-/

If that directory is where the entire project is contained you would think that either projects in the directory would be added again, or you would be able to import from that directory...

Doesn't matter now, trying to reinstall gogs in the same location has not only cleared the database, but that directory as well... Sadly I have lost a shitty project I was working on. Not that I would have completed it but may have needed the code for other project.

@unknwon I use virtualmin, when I select the server and go to "PHP Version" it says "Your system only has PHP version 7.0.27 installed, so selecting a different version for each directory is not possible." - According to that I have PHP >= 5.7

@unknwon
Copy link
Member

unknwon commented May 22, 2018

I don’t know what you saying, PHP is not MySQL.

@ghost
Copy link

ghost commented May 22, 2018

@unknwon Whoops thanks for pointing out I am a stoner without providing steps to locate the correct version number but I am a knowledgeable person so I digress.

I click "Webmin" and then "Servers" and then "MySQL Database Server" and at the top it says "MySQL version 10.1.26" so MySQL >= 5.7

This is all moot anyways, I fixed it and posted how, and this is now an issue of being able to restore a repo located in the folder where the repositories are contained. Annnd that is moot now as well considering my files and database got wiped on attempting a reinstall.

I accept @m2nlight's way in the future to backup and restore, but what then is the point of the repository directory?

@m2nlight
Copy link
Contributor

m2nlight commented May 22, 2018

@MyS5Droid I think you need the gogs-repositories directory and gogs database both. If gogs database is empty, the gogs web UI does not show any repos and others. Because the info is gone. Maybe the conf.ini configured a error database.

@unknwon
Copy link
Member

unknwon commented May 22, 2018

MySQl does not have such version 10.1.26 according to https://en.wikipedia.org/wiki/MySQL#Release_history.

@m2nlight
Copy link
Contributor

upgrade gogs document:

https://gogs.io/docs/upgrade/upgrade_from_binary

@m2nlight
Copy link
Contributor

10.1.26 is MariaDB version. Please update to MariaDB 10.2

@ghost
Copy link

ghost commented Jun 1, 2018 via email

@ghost
Copy link

ghost commented Jun 1, 2018 via email

@m2nlight
Copy link
Contributor

m2nlight commented Jun 1, 2018

@MyS5Droid I am sorry you lost your repos. Restore repos from gogs-repository is a good idea. If you can do that, let's work together to prefect gogs.

@gogs gogs locked and limited conversation to collaborators Jun 2, 2018
@unknwon unknwon removed this from the 0.13 milestone Nov 26, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
💊 bug Something isn't working status: needs feedback Tell me more about it
Projects
None yet
Development

No branches or pull requests