[{"data":1,"prerenderedAt":781},["ShallowReactive",2],{"post-/posts/combining-distinct-and-group-concat-with-custom-delimiters-in-sqlite3":3},{"id":4,"title":5,"body":6,"created":772,"description":773,"extension":774,"meta":775,"navigation":106,"path":776,"seo":777,"stem":778,"updated":779,"__hash__":780},"posts/2.posts/20230517.Combining DISTINCT and group-concat with custom delimiters in SQLite3.md","Combining DISTINCT and group_concat() with custom delimiters in SQLite3",{"type":7,"value":8,"toc":764},"minimark",[9,22,29,32,203,206,224,283,286,291,303,306,322,330,337,371,377,380,384,390,393,409,439,445,461,487,490,494,502,533,539,542,578,584,597,606,641,647,664,670,675,684,736,742,745,749,757,760],[10,11,12,13,17,18,21],"p",{},"In this blog post, I'll introduce two useful tools in SQLite3: ",[14,15,16],"code",{},"group_concat()"," and ",[14,19,20],{},"DISTINCT",".\nI show how they can be used individually, and that you need to watch out when combining them with custom delimiters.",[10,23,24,25,28],{},"I assume you have a basic understanding of SQL, but mostly you will need to know about the concepts of a table with data and simple ",[14,26,27],{},"SELECT"," queries.",[10,30,31],{},"But first, we need a table to work with.\nFor that, we'll create a table of hobbit names, and fill it with a few hobbits from Tolkien's fictional universe:",[33,34,39],"pre",{"className":35,"code":36,"language":37,"meta":38,"style":38},"language-sql shiki shiki-themes one-dark-pro","CREATE TABLE hobbits (\n  first_name VARCHAR(64),\n  last_name VARCHAR(64)\n);\n\nINSERT INTO hobbits VALUES\n  (\"Frodo\", \"Baggins\"),\n  (\"Bilbo\", \"Baggins\"),\n  (\"Sam\", \"Gamgee\"),\n  (\"Pippin\", \"Took\"),\n  (\"Merry\", \"Brandybuck\")\n;\n","sql","",[14,40,41,61,80,95,101,108,120,138,152,167,182,197],{"__ignoreMap":38},[42,43,46,50,53,57],"span",{"class":44,"line":45},"line",1,[42,47,49],{"class":48},"seHd6","CREATE",[42,51,52],{"class":48}," TABLE",[42,54,56],{"class":55},"sVbv2"," hobbits",[42,58,60],{"class":59},"sn6KH"," (\n",[42,62,64,67,70,73,77],{"class":44,"line":63},2,[42,65,66],{"class":59},"  first_name ",[42,68,69],{"class":48},"VARCHAR",[42,71,72],{"class":59},"(",[42,74,76],{"class":75},"sVC51","64",[42,78,79],{"class":59},"),\n",[42,81,83,86,88,90,92],{"class":44,"line":82},3,[42,84,85],{"class":59},"  last_name ",[42,87,69],{"class":48},[42,89,72],{"class":59},[42,91,76],{"class":75},[42,93,94],{"class":59},")\n",[42,96,98],{"class":44,"line":97},4,[42,99,100],{"class":59},");\n",[42,102,104],{"class":44,"line":103},5,[42,105,107],{"emptyLinePlaceholder":106},true,"\n",[42,109,111,114,117],{"class":44,"line":110},6,[42,112,113],{"class":48},"INSERT INTO",[42,115,116],{"class":59}," hobbits ",[42,118,119],{"class":48},"VALUES\n",[42,121,123,126,130,133,136],{"class":44,"line":122},7,[42,124,125],{"class":59},"  (",[42,127,129],{"class":128},"subq3","\"Frodo\"",[42,131,132],{"class":59},", ",[42,134,135],{"class":128},"\"Baggins\"",[42,137,79],{"class":59},[42,139,141,143,146,148,150],{"class":44,"line":140},8,[42,142,125],{"class":59},[42,144,145],{"class":128},"\"Bilbo\"",[42,147,132],{"class":59},[42,149,135],{"class":128},[42,151,79],{"class":59},[42,153,155,157,160,162,165],{"class":44,"line":154},9,[42,156,125],{"class":59},[42,158,159],{"class":128},"\"Sam\"",[42,161,132],{"class":59},[42,163,164],{"class":128},"\"Gamgee\"",[42,166,79],{"class":59},[42,168,170,172,175,177,180],{"class":44,"line":169},10,[42,171,125],{"class":59},[42,173,174],{"class":128},"\"Pippin\"",[42,176,132],{"class":59},[42,178,179],{"class":128},"\"Took\"",[42,181,79],{"class":59},[42,183,185,187,190,192,195],{"class":44,"line":184},11,[42,186,125],{"class":59},[42,188,189],{"class":128},"\"Merry\"",[42,191,132],{"class":59},[42,193,194],{"class":128},"\"Brandybuck\"",[42,196,94],{"class":59},[42,198,200],{"class":44,"line":199},12,[42,201,202],{"class":59},";\n",[10,204,205],{},"To simply select the first and last names of each of the hobbits, the following query can be used:",[33,207,209],{"className":35,"code":208,"language":37,"meta":38,"style":38},"SELECT first_name, last_name FROM hobbits;\n",[14,210,211],{"__ignoreMap":38},[42,212,213,215,218,221],{"class":44,"line":45},[42,214,27],{"class":48},[42,216,217],{"class":59}," first_name, last_name ",[42,219,220],{"class":48},"FROM",[42,222,223],{"class":59}," hobbits;\n",[225,226,227,240],"table",{},[228,229,230],"thead",{},[231,232,233,237],"tr",{},[234,235,236],"th",{},"first_name",[234,238,239],{},"last_name",[241,242,243,252,259,267,275],"tbody",{},[231,244,245,249],{},[246,247,248],"td",{},"Frodo",[246,250,251],{},"Baggins",[231,253,254,257],{},[246,255,256],{},"Bilbo",[246,258,251],{},[231,260,261,264],{},[246,262,263],{},"Sam",[246,265,266],{},"Gamgee",[231,268,269,272],{},[246,270,271],{},"Pippin",[246,273,274],{},"Took",[231,276,277,280],{},[246,278,279],{},"Merry",[246,281,282],{},"Brandybuck",[10,284,285],{},"Great! But we can do more interesting things, as demonstrated next.",[287,288,290],"h2",{"id":289},"concatenating","Concatenating",[10,292,293,294,302],{},"One of the tools in SQL is the ",[295,296,300],"a",{"href":297,"rel":298},"https://www.sqlite.org/lang_aggfunc.html#group_concat",[299],"nofollow",[14,301,16],{}," function, which returns a string that is the concatenation of all non-null values of the given result set.",[10,304,305],{},"For example, this query concatenates all of the hobbits' first names:",[33,307,309],{"className":35,"code":308,"language":37,"meta":38,"style":38},"SELECT group_concat(first_name) FROM hobbits;\n",[14,310,311],{"__ignoreMap":38},[42,312,313,315,318,320],{"class":44,"line":45},[42,314,27],{"class":48},[42,316,317],{"class":59}," group_concat(first_name) ",[42,319,220],{"class":48},[42,321,223],{"class":59},[33,323,328],{"className":324,"code":326,"language":327},[325],"language-text","Frodo,Bilbo,Sam,Pippin,Merry\n","text",[14,329,326],{"__ignoreMap":38},[10,331,332,333,336],{},"You can see that, by default, the values are delimited by commas (",[14,334,335],{},",",").\nTo customise this, it is possible to provide an additional argument that sets the delimiter:",[33,338,340],{"className":35,"code":339,"language":37,"meta":38,"style":38},"SELECT\n  group_concat(first_name, \" and \")\nFROM\n  hobbits\n;\n",[14,341,342,347,357,362,367],{"__ignoreMap":38},[42,343,344],{"class":44,"line":45},[42,345,346],{"class":48},"SELECT\n",[42,348,349,352,355],{"class":44,"line":63},[42,350,351],{"class":59},"  group_concat(first_name, ",[42,353,354],{"class":128},"\" and \"",[42,356,94],{"class":59},[42,358,359],{"class":44,"line":82},[42,360,361],{"class":48},"FROM\n",[42,363,364],{"class":44,"line":97},[42,365,366],{"class":59},"  hobbits\n",[42,368,369],{"class":44,"line":103},[42,370,202],{"class":59},[33,372,375],{"className":373,"code":374,"language":327},[325],"Frodo and Bilbo and Sam and Pippin and Merry\n",[14,376,374],{"__ignoreMap":38},[10,378,379],{},"That's a little more readable.",[287,381,383],{"id":382},"distinct-values","Distinct values",[10,385,386,387,389],{},"Another tool in SQL is the ",[14,388,20],{}," keyword, which can remove duplicate entries from result sets.",[10,391,392],{},"The keen-eyed (or well-read) among you may have observed (or remembered) that Frodo and Bilbo share their last names!\nThis means that when selecting the last names, we would see a duplicate entry.\nAnd indeed:",[33,394,396],{"className":35,"code":395,"language":37,"meta":38,"style":38},"SELECT last_name FROM hobbits;\n",[14,397,398],{"__ignoreMap":38},[42,399,400,402,405,407],{"class":44,"line":45},[42,401,27],{"class":48},[42,403,404],{"class":59}," last_name ",[42,406,220],{"class":48},[42,408,223],{"class":59},[225,410,411,417],{},[228,412,413],{},[231,414,415],{},[234,416,239],{},[241,418,419,423,427,431,435],{},[231,420,421],{},[246,422,251],{},[231,424,425],{},[246,426,251],{},[231,428,429],{},[246,430,266],{},[231,432,433],{},[246,434,274],{},[231,436,437],{},[246,438,282],{},[10,440,441,442,444],{},"To remove the duplicate Bagginses from the result set, you can specify that the result set should be made distinct with the ",[14,443,20],{}," keyword:",[33,446,448],{"className":35,"code":447,"language":37,"meta":38,"style":38},"SELECT DISTINCT last_name FROM hobbits;\n",[14,449,450],{"__ignoreMap":38},[42,451,452,455,457,459],{"class":44,"line":45},[42,453,454],{"class":48},"SELECT DISTINCT",[42,456,404],{"class":59},[42,458,220],{"class":48},[42,460,223],{"class":59},[225,462,463,469],{},[228,464,465],{},[231,466,467],{},[234,468,239],{},[241,470,471,475,479,483],{},[231,472,473],{},[246,474,251],{},[231,476,477],{},[246,478,266],{},[231,480,481],{},[246,482,274],{},[231,484,485],{},[246,486,282],{},[10,488,489],{},"As expected, only a single Baggins remains in the result.",[287,491,493],{"id":492},"concatenating-distinct-values","Concatenating distinct values",[10,495,496,497,17,499,501],{},"So, what if we want to combine ",[14,498,16],{},[14,500,20],{},"?\nWe can!\nThis will concatenate all the distinct last names:",[33,503,505],{"className":35,"code":504,"language":37,"meta":38,"style":38},"SELECT\n  group_concat(DISTINCT last_name)\nFROM\n  hobbits\n;\n",[14,506,507,511,521,525,529],{"__ignoreMap":38},[42,508,509],{"class":44,"line":45},[42,510,346],{"class":48},[42,512,513,516,518],{"class":44,"line":63},[42,514,515],{"class":59},"  group_concat(",[42,517,20],{"class":48},[42,519,520],{"class":59}," last_name)\n",[42,522,523],{"class":44,"line":82},[42,524,361],{"class":48},[42,526,527],{"class":44,"line":97},[42,528,366],{"class":59},[42,530,531],{"class":44,"line":103},[42,532,202],{"class":59},[33,534,537],{"className":535,"code":536,"language":327},[325],"Baggins,Gamgee,Took,Brandybuck\n",[14,538,536],{"__ignoreMap":38},[10,540,541],{},"Fabulous!\nWe're wielding these tools like a pro!\nFeeling overly confident, we may even try to customise the delimiter for extra points:",[33,543,545],{"className":35,"code":544,"language":37,"meta":38,"style":38},"SELECT\n    group_concat(DISTINCT last_name, \" and \")\nFROM\n    hobbits\n;\n",[14,546,547,551,565,569,574],{"__ignoreMap":38},[42,548,549],{"class":44,"line":45},[42,550,346],{"class":48},[42,552,553,556,558,561,563],{"class":44,"line":63},[42,554,555],{"class":59},"    group_concat(",[42,557,20],{"class":48},[42,559,560],{"class":59}," last_name, ",[42,562,354],{"class":128},[42,564,94],{"class":59},[42,566,567],{"class":44,"line":82},[42,568,361],{"class":48},[42,570,571],{"class":44,"line":97},[42,572,573],{"class":59},"    hobbits\n",[42,575,576],{"class":44,"line":103},[42,577,202],{"class":59},[33,579,582],{"className":580,"code":581,"language":327},[325],"Parse error near line 2: DISTINCT aggregates must\n  have exactly one argument\n",[14,583,581],{"__ignoreMap":38},[10,585,586,587,589,590,593,594,596],{},"Uh oh.\nIt looks like the way we use the ",[14,588,20],{}," keyword confuses the query parser.\nIt seems like the ",[14,591,592],{},"last_name, \" and \""," part is interpreted as two arguments for the ",[14,595,20],{}," keyword?",[10,598,599,600,602,603,605],{},"Let's try to fix that with parentheses so that the ",[14,601,20],{}," keyword only applies to ",[14,604,239],{},":",[33,607,609],{"className":35,"code":608,"language":37,"meta":38,"style":38},"SELECT\n    group_concat((DISTINCT last_name), \" and \")\nFROM\n    hobbits\n;\n",[14,610,611,615,629,633,637],{"__ignoreMap":38},[42,612,613],{"class":44,"line":45},[42,614,346],{"class":48},[42,616,617,620,622,625,627],{"class":44,"line":63},[42,618,619],{"class":59},"    group_concat((",[42,621,20],{"class":48},[42,623,624],{"class":59}," last_name), ",[42,626,354],{"class":128},[42,628,94],{"class":59},[42,630,631],{"class":44,"line":82},[42,632,361],{"class":48},[42,634,635],{"class":44,"line":97},[42,636,573],{"class":59},[42,638,639],{"class":44,"line":103},[42,640,202],{"class":59},[33,642,645],{"className":643,"code":644,"language":327},[325],"Parse error near line 39: near \"DISTINCT\": syntax\n  error\n  SELECT     group_concat((DISTINCT last_name),\n    \" and \") FROM     hobbits ;\n             error here ---^\n",[14,646,644],{"__ignoreMap":38},[10,648,649,650,657,658,660,661,663],{},"Nope! It seems like this is not allowed.\nConsulting the ",[295,651,654,656],{"href":652,"rel":653},"https://www.sqlite.org/syntax/select-stmt.html",[299],[14,655,27],{}," grammar",", it is indeed the case that it is required for the ",[14,659,20],{}," keyword to directly follow the opening parenthesis of the ",[14,662,16],{}," (or other) function.",[10,665,666,667,669],{},"It looks like the ",[14,668,20],{}," keyword is only allowed in certain contexts, and a special case has been made for adding it in function invocations.\nThis special case prohibits extra arguments in the function invocation.",[671,672,674],"h3",{"id":673},"the-solution","The Solution",[10,676,677,678,683],{},"So, how do we solve this?\nWell, ",[295,679,682],{"href":680,"rel":681},"https://sqlite.org/forum/info/221c2926f5e6f155",[299],"an answer on the SQLite Forum"," provided a solution!\nIt uses a subquery as a workaround:",[33,685,687],{"className":35,"code":686,"language":37,"meta":38,"style":38},"SELECT\n  group_concat(distinct_last_name, \" and \")\nFROM (\n  SELECT DISTINCT\n    last_name AS distinct_last_name\n  FROM hobbits\n);\n",[14,688,689,693,702,708,713,724,732],{"__ignoreMap":38},[42,690,691],{"class":44,"line":45},[42,692,346],{"class":48},[42,694,695,698,700],{"class":44,"line":63},[42,696,697],{"class":59},"  group_concat(distinct_last_name, ",[42,699,354],{"class":128},[42,701,94],{"class":59},[42,703,704,706],{"class":44,"line":82},[42,705,220],{"class":48},[42,707,60],{"class":59},[42,709,710],{"class":44,"line":97},[42,711,712],{"class":48},"  SELECT DISTINCT\n",[42,714,715,718,721],{"class":44,"line":103},[42,716,717],{"class":59},"    last_name ",[42,719,720],{"class":48},"AS",[42,722,723],{"class":59}," distinct_last_name\n",[42,725,726,729],{"class":44,"line":110},[42,727,728],{"class":48},"  FROM",[42,730,731],{"class":59}," hobbits\n",[42,733,734],{"class":44,"line":122},[42,735,100],{"class":59},[33,737,740],{"className":738,"code":739,"language":327},[325],"Baggins and Gamgee and Took and Brandybuck\n",[14,741,739],{"__ignoreMap":38},[10,743,744],{},"The inner query creates an intermediate result set that we can use in the outer query.\nThis workaround definitely feels like... well, a workaround.\nIf you find a more elegant solution to this, please let me know!",[287,746,748],{"id":747},"conclusion","Conclusion",[10,750,751,752,17,754,756],{},"Apparently, the ",[14,753,16],{},[14,755,20],{}," tools do not work together as I expected when using a custom delimiter.",[10,758,759],{},"As Bilbo Baggins would put it: I don't know half of SQL as well as I should like; and I like less than half of it half as well as it deserves.",[761,762,763],"style",{},"html pre.shiki code .seHd6, html code.shiki .seHd6{--shiki-default:#C678DD}html pre.shiki code .sVbv2, html code.shiki .sVbv2{--shiki-default:#61AFEF}html pre.shiki code .sn6KH, html code.shiki .sn6KH{--shiki-default:#ABB2BF}html pre.shiki code .sVC51, html code.shiki .sVC51{--shiki-default:#D19A66}html pre.shiki code .subq3, html code.shiki .subq3{--shiki-default:#98C379}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}",{"title":38,"searchDepth":63,"depth":63,"links":765},[766,767,768,771],{"id":289,"depth":63,"text":290},{"id":382,"depth":63,"text":383},{"id":492,"depth":63,"text":493,"children":769},[770],{"id":673,"depth":82,"text":674},{"id":747,"depth":63,"text":748},"2023-05-17","How to use DISTINCT and group_concat() with custom delimiters in SQLite3.","md",{},"/posts/combining-distinct-and-group-concat-with-custom-delimiters-in-sqlite3",{"title":5,"description":773},"2.posts/20230517.Combining DISTINCT and group-concat with custom delimiters in SQLite3",null,"6mL4W5qn2rBDyftraL23X546533kQlnUKg-0wE5tERY",1775234851922]