How do I install sqlite3 gem on an M1 Mac?

Question

I am trying to install the sqlite3-ruby gem on my Mac, which is running an M1 Pro chip.

The installation fails with the following output:

Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    current directory: /Users/jay/.frum/versions/3.1.0/lib/ruby/gems/3.1.0/gems/sqlite3-1.4.2/ext/sqlite3
/Users/jay/.frum/versions/3.1.0/bin/ruby -I /Users/jay/.frum/versions/3.1.0/lib/ruby/3.1.0 -r ./siteconf20220215-37077-7vshdb.rb extconf.rb
checking for sqlite3.h... yes
checking for pthread_create() in -lpthread... yes
checking for -ldl... yes
checking for sqlite3_libversion_number() in -lsqlite3... no
sqlite3 is missing. Try 'brew install sqlite3',
'yum install sqlite-devel' or 'apt-get install libsqlite3-dev'
and check your shared library search path (the
location where your sqlite3 shared library is located).
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Looking at the mkmf.log file, there are a number of errors that seem to be related to compilation process building for the wrong architecture:

DYLD_FALLBACK_LIBRARY_PATH=.:/Users/jay/.frum/versions/3.1.0/lib pkg-config --exists sqlite3
DYLD_FALLBACK_LIBRARY_PATH=.:/Users/jay/.frum/versions/3.1.0/lib pkg-config --libs sqlite3 |
=> "-L/opt/homebrew/Cellar/sqlite/3.37.2/lib -lsqlite3\n"
DYLD_FALLBACK_LIBRARY_PATH=.:/Users/jay/.frum/versions/3.1.0/lib "clang -o conftest -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0/arm64-darwin21 -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0/ruby/backward -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0 -I. -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT   -fdeclspec -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wdeprecated-declarations -Wdivision-by-zero -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wshorten-64-to-32 -Wwrite-strings -Wold-style-definition -Wmissing-noreturn -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wunused-variable -Wextra-tokens -Wundef  -pipe conftest.c  -L. -L/Users/jay/.frum/versions/3.1.0/lib -L. -fstack-protector-strong -L/usr/local/lib     -lruby.3.1-static -framework CoreFoundation -lpthread -ldl -lobjc   "
ld: warning: ignoring file /Users/jay/.frum/versions/3.1.0/lib/libruby.3.1-static.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

DYLD_FALLBACK_LIBRARY_PATH=.:/Users/jay/.frum/versions/3.1.0/lib "clang -o conftest -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0/arm64-darwin21 -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0/ruby/backward -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0 -I. -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT   -fdeclspec -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wdeprecated-declarations -Wdivision-by-zero -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wshorten-64-to-32 -Wwrite-strings -Wold-style-definition -Wmissing-noreturn -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wunused-variable -Wextra-tokens -Wundef  -pipe conftest.c  -L. -L/Users/jay/.frum/versions/3.1.0/lib -L. -fstack-protector-strong -L/usr/local/lib     -lruby.3.1-static -framework CoreFoundation -lpthread -ldl -lobjc -L/opt/homebrew/Cellar/sqlite/3.37.2/lib -lsqlite3  "
ld: warning: ignoring file /Users/jay/.frum/versions/3.1.0/lib/libruby.3.1-static.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /opt/homebrew/Cellar/sqlite/3.37.2/lib/libsqlite3.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

DYLD_FALLBACK_LIBRARY_PATH=.:/Users/jay/.frum/versions/3.1.0/lib pkg-config --cflags-only-I sqlite3 |
=> "-I/opt/homebrew/Cellar/sqlite/3.37.2/include\n"
DYLD_FALLBACK_LIBRARY_PATH=.:/Users/jay/.frum/versions/3.1.0/lib pkg-config --cflags-only-other sqlite3 |
=> "\n"
DYLD_FALLBACK_LIBRARY_PATH=.:/Users/jay/.frum/versions/3.1.0/lib pkg-config --libs-only-l sqlite3 |
=> "-lsqlite3\n"
package configuration for sqlite3
incflags: -I/opt/homebrew/Cellar/sqlite/3.37.2/include
cflags: 
ldflags: -L/opt/homebrew/Cellar/sqlite/3.37.2/lib
libs: -lsqlite3

find_header: checking for sqlite3.h... -------------------- yes

DYLD_FALLBACK_LIBRARY_PATH=.:/Users/jay/.frum/versions/3.1.0/lib "clang -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0/arm64-darwin21 -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0/ruby/backward -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0 -I. -I/opt/homebrew/Cellar/sqlite/3.37.2/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT   -fdeclspec -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wdeprecated-declarations -Wdivision-by-zero -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wshorten-64-to-32 -Wwrite-strings -Wold-style-definition -Wmissing-noreturn -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wunused-variable -Wextra-tokens -Wundef  -pipe    -c conftest.c"
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: #include <sqlite3.h>
/* end */

--------------------

find_library: checking for pthread_create() in -lpthread... -------------------- yes

DYLD_FALLBACK_LIBRARY_PATH=.:/Users/jay/.frum/versions/3.1.0/lib "clang -o conftest -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0/arm64-darwin21 -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0/ruby/backward -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0 -I. -I/opt/homebrew/Cellar/sqlite/3.37.2/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT   -fdeclspec -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wdeprecated-declarations -Wdivision-by-zero -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wshorten-64-to-32 -Wwrite-strings -Wold-style-definition -Wmissing-noreturn -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wunused-variable -Wextra-tokens -Wundef  -pipe  conftest.c  -L. -L/Users/jay/.frum/versions/3.1.0/lib -L. -fstack-protector-strong -L/usr/local/lib -L/opt/homebrew/Cellar/sqlite/3.37.2/lib     -lsqlite3 -lruby.3.1-static -framework CoreFoundation -lpthread -ldl -lobjc -lpthread  -lsqlite3  "
conftest.c:14:57: error: use of undeclared identifier 'pthread_create'
int t(void) { void ((*volatile p)()); p = (void ((*)()))pthread_create; return !p; }
                                                        ^
1 error generated.
checked program was:
/* begin */
 1: #include "ruby.h"
 2: 
 3: /*top*/
 4: extern int t(void);
 5: int main(int argc, char **argv)
 6: {
 7:   if (argc > 1000000) {
 8:     int (* volatile tp)(void)=(int (*)(void))&t;
 9:     printf("%d", (*tp)());
10:   }
11: 
12:   return !!argv[argc];
13: }
14: int t(void) { void ((*volatile p)()); p = (void ((*)()))pthread_create; return !p; }
/* end */

DYLD_FALLBACK_LIBRARY_PATH=.:/Users/jay/.frum/versions/3.1.0/lib "clang -o conftest -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0/arm64-darwin21 -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0/ruby/backward -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0 -I. -I/opt/homebrew/Cellar/sqlite/3.37.2/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT   -fdeclspec -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wdeprecated-declarations -Wdivision-by-zero -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wshorten-64-to-32 -Wwrite-strings -Wold-style-definition -Wmissing-noreturn -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wunused-variable -Wextra-tokens -Wundef  -pipe  conftest.c  -L. -L/Users/jay/.frum/versions/3.1.0/lib -L. -fstack-protector-strong -L/usr/local/lib -L/opt/homebrew/Cellar/sqlite/3.37.2/lib     -lsqlite3 -lruby.3.1-static -framework CoreFoundation -lpthread -ldl -lobjc -lpthread  -lsqlite3  "
ld: warning: ignoring file /opt/homebrew/Cellar/sqlite/3.37.2/lib/libsqlite3.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/jay/.frum/versions/3.1.0/lib/libruby.3.1-static.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
checked program was:
/* begin */
 1: #include "ruby.h"
 2: 
 3: /*top*/
 4: extern int t(void);
 5: int main(int argc, char **argv)
 6: {
 7:   if (argc > 1000000) {
 8:     int (* volatile tp)(void)=(int (*)(void))&t;
 9:     printf("%d", (*tp)());
10:   }
11: 
12:   return !!argv[argc];
13: }
14: extern void pthread_create();
15: int t(void) { pthread_create(); return 0; }
/* end */

--------------------

have_library: checking for -ldl... -------------------- yes

DYLD_FALLBACK_LIBRARY_PATH=.:/Users/jay/.frum/versions/3.1.0/lib "clang -o conftest -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0/arm64-darwin21 -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0/ruby/backward -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0 -I. -I/opt/homebrew/Cellar/sqlite/3.37.2/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT   -fdeclspec -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wdeprecated-declarations -Wdivision-by-zero -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wshorten-64-to-32 -Wwrite-strings -Wold-style-definition -Wmissing-noreturn -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wunused-variable -Wextra-tokens -Wundef  -pipe  conftest.c  -L. -L/Users/jay/.frum/versions/3.1.0/lib -L. -fstack-protector-strong -L/usr/local/lib -L/opt/homebrew/Cellar/sqlite/3.37.2/lib    -lpthread  -lsqlite3 -lruby.3.1-static -framework CoreFoundation -lpthread -ldl -lobjc -ldl -lpthread  -lsqlite3  "
ld: warning: ignoring file /opt/homebrew/Cellar/sqlite/3.37.2/lib/libsqlite3.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/jay/.frum/versions/3.1.0/lib/libruby.3.1-static.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
checked program was:
/* begin */
 1: #include "ruby.h"
 2: 
 3: /*top*/
 4: extern int t(void);
 5: int main(int argc, char **argv)
 6: {
 7:   if (argc > 1000000) {
 8:     int (* volatile tp)(void)=(int (*)(void))&t;
 9:     printf("%d", (*tp)());
10:   }
11: 
12:   return !!argv[argc];
13: }
14: 
15: int t(void) { ; return 0; }
/* end */

--------------------

find_library: checking for sqlite3_libversion_number() in -lsqlite3... -------------------- no

DYLD_FALLBACK_LIBRARY_PATH=.:/Users/jay/.frum/versions/3.1.0/lib "clang -o conftest -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0/arm64-darwin21 -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0/ruby/backward -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0 -I. -I/opt/homebrew/Cellar/sqlite/3.37.2/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT   -fdeclspec -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wdeprecated-declarations -Wdivision-by-zero -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wshorten-64-to-32 -Wwrite-strings -Wold-style-definition -Wmissing-noreturn -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wunused-variable -Wextra-tokens -Wundef  -pipe  conftest.c  -L. -L/Users/jay/.frum/versions/3.1.0/lib -L. -fstack-protector-strong -L/usr/local/lib -L/opt/homebrew/Cellar/sqlite/3.37.2/lib    -ldl -lpthread  -lsqlite3 -lruby.3.1-static -framework CoreFoundation -lpthread -ldl -lobjc -lsqlite3 -ldl -lpthread  -lsqlite3  "
conftest.c:14:57: error: use of undeclared identifier 'sqlite3_libversion_number'
int t(void) { void ((*volatile p)()); p = (void ((*)()))sqlite3_libversion_number; return !p; }
                                                        ^
1 error generated.
checked program was:
/* begin */
 1: #include "ruby.h"
 2: 
 3: /*top*/
 4: extern int t(void);
 5: int main(int argc, char **argv)
 6: {
 7:   if (argc > 1000000) {
 8:     int (* volatile tp)(void)=(int (*)(void))&t;
 9:     printf("%d", (*tp)());
10:   }
11: 
12:   return !!argv[argc];
13: }
14: int t(void) { void ((*volatile p)()); p = (void ((*)()))sqlite3_libversion_number; return !p; }
/* end */

DYLD_FALLBACK_LIBRARY_PATH=.:/Users/jay/.frum/versions/3.1.0/lib "clang -o conftest -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0/arm64-darwin21 -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0/ruby/backward -I/Users/jay/.frum/versions/3.1.0/include/ruby-3.1.0 -I. -I/opt/homebrew/Cellar/sqlite/3.37.2/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT   -fdeclspec -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wdeprecated-declarations -Wdivision-by-zero -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wshorten-64-to-32 -Wwrite-strings -Wold-style-definition -Wmissing-noreturn -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wunused-variable -Wextra-tokens -Wundef  -pipe  conftest.c  -L. -L/Users/jay/.frum/versions/3.1.0/lib -L. -fstack-protector-strong -L/usr/local/lib -L/opt/homebrew/Cellar/sqlite/3.37.2/lib    -ldl -lpthread  -lsqlite3 -lruby.3.1-static -framework CoreFoundation -lpthread -ldl -lobjc -lsqlite3 -ldl -lpthread  -lsqlite3  "
ld: warning: ignoring file /opt/homebrew/Cellar/sqlite/3.37.2/lib/libsqlite3.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Users/jay/.frum/versions/3.1.0/lib/libruby.3.1-static.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
Undefined symbols for architecture x86_64:
  "_sqlite3_libversion_number", referenced from:
      _t in conftest-593102.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
checked program was:
/* begin */
 1: #include "ruby.h"
 2: 
 3: /*top*/
 4: extern int t(void);
 5: int main(int argc, char **argv)
 6: {
 7:   if (argc > 1000000) {
 8:     int (* volatile tp)(void)=(int (*)(void))&t;
 9:     printf("%d", (*tp)());
10:   }
11: 
12:   return !!argv[argc];
13: }
14: extern void sqlite3_libversion_number();
15: int t(void) { sqlite3_libversion_number(); return 0; }
/* end */

--------------------

I installed sqlite3 via Homebrew, with the following output:

sqlite: stable 3.37.2 (bottled) [keg-only]
Command-line interface for SQLite
https://sqlite.org/index.html
/opt/homebrew/Cellar/sqlite/3.37.2 (11 files, 4.3MB)
  Poured from bottle on 2022-02-15 at 13:01:01
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/sqlite.rb
License: blessing
==> Dependencies
Required: readline ✔
==> Caveats
sqlite is keg-only, which means it was not symlinked into /opt/homebrew,
because macOS already provides this software and installing another version in
parallel can cause all kinds of trouble.

If you need to have sqlite first in your PATH, run:
  echo 'export PATH="/opt/homebrew/opt/sqlite/bin:$PATH"' >> ~/.zshrc

For compilers to find sqlite you may need to set:
  export LDFLAGS="-L/opt/homebrew/opt/sqlite/lib"
  export CPPFLAGS="-I/opt/homebrew/opt/sqlite/include"

For pkg-config to find sqlite you may need to set:
  export PKG_CONFIG_PATH="/opt/homebrew/opt/sqlite/lib/pkgconfig"

Taking the advice from Homebrew, I set bundle config build.sqlite3 --with-sqlite3-include=$(brew --prefix sqlite)/include --with-sqlite3-lib=$(brew --prefix sqlite)/lib but I got what appears to be the same result.

Can somebody please tell me what's the correct way to install this gem given this setup? Thank you very much.

Answer 1

In case anyone else stumbles along this, I had a similar issue. I have my dev environment set up part emulated and part ARM.

To navigate this, I have two different versions of Homebrew installed. One version is for M1 architecture, which lives in /opt/homebrew/. And I have a version of Homebrew that runs on x86 architecture that lives in /usr/local/Homebrew.

I have the x86 version aliased to ibrew, like this:

alias ibrew='arch --x86_64 /usr/local/Homebrew/bin/brew'

(See this answer for more info on running two separate Homebrew versions.)

OP's bundle config command worked for me, I just had to swap out brew for ibrew:

bundle config build.sqlite3 --with-sqlite3-include=$(brew --prefix sqlite)/include --with-sqlite3-lib=$(brew --prefix sqlite)/lib