Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is there any way we can run multiple SQL queries at the same time in Athena Database

I have to create 20 table in a Athena data base at the same time. Can I do it with a single execution.

example :

CREATE EXTERNAL TABLE IF NOT EXISTS database_1.A 
; 
CREATE EXTERNAL TABLE IF NOT EXISTS database_1.B
;
CREATE EXTERNAL TABLE IF NOT EXISTS database_1.C
like image 947
bibin jose Avatar asked Nov 16 '25 10:11

bibin jose


2 Answers

I have used aws cli for such problems. create a list of sqls.

sql_list.txt
CREATE EXTERNAL TABLE IF NOT EXISTS database_1.A; 
CREATE EXTERNAL TABLE IF NOT EXISTS database_1.B;
CREATE EXTERNAL TABLE IF NOT EXISTS database_1.C;
----------
exec_sqls.sh
input_file=$1
while IFS= read -r sql
do
  echo "$line"
  aws athena start-query-execution --query-string "$sql" --result-configuration S3LocationForOutput=s3://<bucket>
done < "$input_file"
-----------
sh -x exec_sqls.sh sql_list.txt
like image 86
pyBomb Avatar answered Nov 18 '25 07:11

pyBomb


You can submit multiple requests simultaneously to Amazon Athena (eg via different threads in your application), but each Amazon Athena command can only execute a single SQL query/command.

like image 38
John Rotenstein Avatar answered Nov 18 '25 08:11

John Rotenstein



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!