快速查询以对SQL数据进行规范化 [英] Fast query to do normalization on SQL data

问题描述

我有一些要标准化的数据。具体来说，我正在对其进行归一化，这样我就可以处理正在归一化的部分，而不必担心重复。我正在做的是：

I have some data that I want to normalize. Specifically I'm normalizing it so I can process the portions getting normalized without having to worry about duplicates. What I'm doing is:

INSERT INTO new_table (a, b, c)
    SELECT DISTINCT a,b,c
    FROM old_table;

UPDATE old_table
SET abc_id = new_table.id
FROM new_table
WHERE new_table.a = old_table.a 
  AND new_table.b = old_table.b  
  AND new_table.c = old_table.c;

首先，似乎应该有一个更好的方法。看起来，查找不同数据的固有过程可能会产生一个属于它的成员的列表。其次，更重要的是，INSERT需要花费几个时间，而UPDATE需要 FOREVER （我实际上没有知道需要花费多长时间的值，因为它仍在运行）。我正在使用postgresql。有没有更好的方法（也许所有查询都在其中）。

First off, it seems as if there should be a better way of doing this. It seems that the inherent process of finding the distinct data could produce a list of the members that belong to it. Second, and more important, the INSERT takes a couple and the UPDATE takes FOREVER (I don't actually have a value for how long it takes yet because it's still running). I'm using postgresql. Is there a better way of doing this (perhaps all in one query).

推荐答案

这是我的另一个答案，扩展为三个列：

This is my other answer, extended to three columns:

        -- Some test data
CREATE TABLE the_table
        ( id SERIAL NOT NULL PRIMARY KEY
        , name varchar
        , a INTEGER
        , b varchar
        , c varchar
        );
INSERT INTO the_table(name, a,b,c) VALUES
 ( 'Chimpanzee' , 1, 'mammals', 'apes' )
,( 'Urang Utang' , 1, 'mammals', 'apes' )
,( 'Homo Sapiens' , 1, 'mammals', 'apes' )
,( 'Mouse' , 2, 'mammals', 'rodents' )
,( 'Rat' , 2, 'mammals', 'rodents' )
,( 'Cat' , 3, 'mammals', 'felix' )
,( 'Dog' , 3, 'mammals', 'canae' )
        ;

        -- [empty] table to contain the "squeezed out" domain {a,b,c}
CREATE TABLE abc_table
        ( id SERIAL NOT NULL PRIMARY KEY
        , a INTEGER
        , b varchar
        , c varchar
        , UNIQUE (a,b,c)
        );

        -- The original table needs a "link" to the new table
ALTER TABLE the_table
        ADD column abc_id INTEGER -- NOT NULL
        REFERENCES abc_table(id)
        ;
        -- FK constraints are helped a lot by a supportive index.
CREATE INDEX abc_table_fk ON the_table (abc_id);

        -- Chained query to:
        -- * populate the domain table
        -- * initialize the FK column in the original table
WITH ins AS (
        INSERT INTO abc_table(a,b,c)
        SELECT DISTINCT a,b,c
        FROM the_table a
        RETURNING *
        )
UPDATE the_table ani
SET abc_id = ins.id
FROM ins
WHERE ins.a = ani.a
AND ins.b = ani.b
AND ins.c = ani.c
        ;

        -- Now that we have the FK pointing to the new table,
        -- we can drop the redundant columns.
ALTER TABLE the_table DROP COLUMN a, DROP COLUMN b, DROP COLUMN c;

SELECT * FROM the_table;
SELECT * FROM abc_table;

        -- show it to the world
SELECT a.*
        , c.a, c.b, c.c
FROM the_table a
JOIN abc_table c ON c.id = a.abc_id
        ;

---

结果：

---

Results:

CREATE TABLE
INSERT 0 7
CREATE TABLE
ALTER TABLE
CREATE INDEX
UPDATE 7
ALTER TABLE
 id |     name     | abc_id 
----+--------------+--------
  1 | Chimpanzee   |      4
  2 | Urang Utang  |      4
  3 | Homo Sapiens |      4
  4 | Mouse        |      3
  5 | Rat          |      3
  6 | Cat          |      1
  7 | Dog          |      2
(7 rows)

 id | a |    b    |    c    
----+---+---------+---------
  1 | 3 | mammals | felix
  2 | 3 | mammals | canae
  3 | 2 | mammals | rodents
  4 | 1 | mammals | apes
(4 rows)

 id |     name     | abc_id | a |    b    |    c    
----+--------------+--------+---+---------+---------
  1 | Chimpanzee   |      4 | 1 | mammals | apes
  2 | Urang Utang  |      4 | 1 | mammals | apes
  3 | Homo Sapiens |      4 | 1 | mammals | apes
  4 | Mouse        |      3 | 2 | mammals | rodents
  5 | Rat          |      3 | 2 | mammals | rodents
  6 | Cat          |      1 | 3 | mammals | felix
  7 | Dog          |      2 | 3 | mammals | canae
(7 rows)

编辑：这似乎效果很好，我讨厌看到我放在那里的票数低了，所以没用编辑（CrazyCasta）。

This seems to work well enough and I hate to see the down-vote I put there, so useless edit (CrazyCasta).

这篇关于快速查询以对SQL数据进行规范化的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！