VACUUM显着降低了性能。数据库变得无法使用! [英] VACUUM degrades performance significantly. Database becomes unusable!

查看:125
本文介绍了VACUUM显着降低了性能。数据库变得无法使用!的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

你好,


对于大型桌面上的普通VACUUM来说,降低性能是否正常超过
9倍?当VACUUM运行时,我的数据库变得无法使用。从阅读
新闻组开始,我认为VACUUM应该只会减慢10%到15%。其他MVCC

数据库(如MySQL InnoDB)甚至可以离散地进行VACUUM(在内部运行)。

我的Linux系统还是PostgreSQL?


数据库主要是只读的。有133,000行,每行是大约2.5kB的
(主要是由于bytea列保存了二进制图像)。

长行导致系统转为TOAST表。 VACUUM需要5分20秒才能完成
。系统空转时我反复运行以下测试:

正常运行时:

=================== =

tsdb =#explain analyze select * from table1 where id =

''33a4e9b6eae09634f4ff3e6fa9280f6e'';

QUERY PLAN

----------------------------------------------- -----------------------------

------------- ---------------------------------------

使用索引扫描table1_pkey on table1(cost = 0.00..6.01 rows = 1 width = 346)

(实际时间= 25.30..25.31行= 1循环= 1)

Index Cond :(id =''33a4e9b6eae09634f4ff3e6fa9280f6e''::字符不同)

总运行时间:25.52毫秒

(3行)


当VACUUM运行时:

=================

tsdb =#explain analyze select * from table1 where id =

''336139b47b7faf09fc4d4f03680a4ce5'';

查询计划

------------------ ------------------------------------------------- ---------

--------------------------------- -----------------------

使用table1上的table1_pkey进行索引扫描(成本= 0.00..6.01行= 1宽度= 346 )

(实际时间= 2290.07..2290.10行= 1循环= 1)

指数条件:(id =''336139b47b7faf09fc4d4f03680a4ce5''::字符变化)

总运行时间:2290.22毫秒

(3行)

VACUUM输出:

======== ======

tsdb =#VACUUM VERBOSE table1;

INFO: - Relation public.table1--

INFO:Pages 5887:改为0,空0; Tup 132672:Vac 0,Keep 0,UnUsed

144.

总CPU 0.28s / 0.01u秒过去36.08秒

信息: --Relation pg_toast.pg_toast_12437088--

INFO:40495页:更改0,空0; Tup 197587:Vac 0,保持0,UnUsed

235.

总CPU 1.73s / 0.20u秒过去233.91秒。

VACUUM <当VACUUM'时,
vmstat:

========================

触发内存交换io系统

cpu

rbw swpd free buff cache si所以bi bo in cs us sy

id

0 1 1 74420 6520 30616 405128 0 0 1280 0 287 487 0 1

99

0 1 0 74420 6520 30620 405168 0 0 1196 0 271 436 0 0

100

0 1 1 74420 6520 30620 405120 0 0 1496 4 289 491 0 3

97

0 1 1 74420 6520 30620 405208 0 0 1280 0 268 466 0 0



1 0 1 74420 6520 30620 405208 0 0 1280 0 288 482 0 1

99

1 0 1 74420 6520 30632 405200 0 0 1416 8 277 441 1 2

97

3 1 1 74416 6520 30632 405196 4 0 1284 0 284 473 0 3

97


PostgreSQL配置(唯一)所做的更改):

====================================== ===========

max_connections = 1024

shared_buffers = 2800

sort_mem = 8192

vacuum_mem = 8192

effective_cache_size = 32000

系统:

=======

硬件:AMD 1.2GHz Athlon 512MB SDRAM

操作系统:Redhat Linux 9.0(内核2.4.20-8)

FS:EXT3带有Jatialling,带有noatime,UDMA5

Disk1(Linux):IDE 20GB 7200 RPM Western Digital

Disk2(PostgreSQL):IDE 120GB 7200 RPM Western Digital

PostgreSQL:7.3.4

hdparm:

=======

/ dev / hda :( Linux分区)

multcount = 16 (上)

IO_support = 1(32位)

unmaskirq = 1(上)

using_dma = 1(on)

keepsettings = 0 (关闭)

readonly = 0(关闭)

readahead = 8(开启)

geometry = 2498/255/63, sectors = 40132503 ,start = 0

/ dev / hdc :( PostgreSQL分区)

multcount = 16(on)

IO_support = 1 (32位)

unmaskirq = 1(上)

using_dma = 1(上)

keepsettings = 1(on)

readonly = 0(关闭)

readahead = 8(on)

geometry = 232581/16/63, sectors = 234441648,start = 0

架构:

=======

CREATE TABLE table1(

id varchar(32)DEFAULT' '''NOT NULL,

colname1 varchar(10)DEFAULT''http''NOT

NULL,

colname2 varchar(300)DEFAULT ''''NOT NULL,

colname3 varchar(5)DEFAULT''''NOT

NULL,

c olname4 varchar(300)DEFAULT''''NOT

NULL,

colname5 varchar(300)DEFAULT''''NOT NULL,

colname6 integer DEFAULT 0 NOT

NULL,

colname7 integer DEFAULT 0 NOT

NULL,

colname8 integer DEFAULT 200 NOT NULL,

colname9 varchar(10)DEFAULT''''NOT

NULL,

colname10 varchar(10)DEFAULT'''' NOT NULL,

colname11 varchar(100)DEFAULT''''NOT

NULL,

colname12 varchar(100)DEFAULT'''' NOT NULL,

colname13 varchar(100)DEFAULT''''NOT NULL,

colname14 varchar(20) DEFAULT''''NOT NULL,

colname15 integer DEFAULT 640 NOT

NULL,

colname16 integer DEFAULT 480 NOT

NULL,

colname17 integer DEFAULT 120 NOT

NULL,

colname18 integer DEFAULT 90 NOT

NULL,

colname19 timestamp DEFAULT CURRENT_TIMESTAMP NOT

NULL,

colname20时间戳默认CURRENT_TIMESTAMP NOT

NULL,

colname21 integer DEFAULT 0 NOT NULL,

colname22 integer DEFAULT 0 NOT NULL,

colname23 timestamp DEFAULT CURRENT_TIMESTAMP NOT NULL,

colname24整数默认值0 NOT

NULL,

colname25 integer DEFAULT 0 NOT NULL,

colname26 varchar(10)DEFAULT''''NOT

NULL,

colname28 varchar (10)DEFAULT''''NOT NULL,

colname29 varchar(10)DEFAULT''jpeg''NOT

NULL,

colname30 varchar(20)DEFAULT''''NOT NULL,

colname31 bytea,

PRIMARY KEY(id)

)没有OIDS


CREATE UNIQUE INDEX table1_idx_1 ON table1(colname1,colname2,colname3,

colname4,colname5)


叹气,: - (


Stephen

Hello,

Is it normal for plain VACUUM on large table to degrade performance by over
9 times? My database becomes unusable when VACUUM runs. From reading
newsgroups, I thought VACUUM should only slow down by 10% to 15%. Other MVCC
databases like MySQL InnoDB can even VACUUM discretely (runs internally). Is
it my Linux system or is it PostgreSQL?

The database is mostly read-only. There are 133,000 rows and each row is
about 2.5kB in size (mostly due to the bytea column holding a binary image).
The long row causes system to TOAST the table. VACUUM takes 5m20s to
complete. I repeatedly ran the following tests while system is idling:
In normal operation:
====================
tsdb=# explain analyze select * from table1 where id =
''33a4e9b6eae09634f4ff3e6fa9280f6e'';
QUERY PLAN
----------------------------------------------------------------------------
----------------------------------------------------
Index Scan using table1_pkey on table1 (cost=0.00..6.01 rows=1 width=346)
(actual time=25.30..25.31 rows=1 loops=1)
Index Cond: (id = ''33a4e9b6eae09634f4ff3e6fa9280f6e''::character varying)
Total runtime: 25.52 msec
(3 rows)

When VACUUM runs:
=================
tsdb=# explain analyze select * from table1 where id =
''336139b47b7faf09fc4d4f03680a4ce5'';
QUERY PLAN
----------------------------------------------------------------------------
--------------------------------------------------------
Index Scan using table1_pkey on table1 (cost=0.00..6.01 rows=1 width=346)
(actual time=2290.07..2290.10 rows=1 loops=1)
Index Cond: (id = ''336139b47b7faf09fc4d4f03680a4ce5''::character varying)
Total runtime: 2290.22 msec
(3 rows)
VACUUM output:
==============
tsdb=# VACUUM VERBOSE table1;
INFO: --Relation public.table1--
INFO: Pages 5887: Changed 0, Empty 0; Tup 132672: Vac 0, Keep 0, UnUsed
144.
Total CPU 0.28s/0.01u sec elapsed 36.08 sec.
INFO: --Relation pg_toast.pg_toast_12437088--
INFO: Pages 40495: Changed 0, Empty 0; Tup 197587: Vac 0, Keep 0, UnUsed
235.
Total CPU 1.73s/0.20u sec elapsed 233.91 sec.
VACUUM

vmstat while VACUUM''ing:
========================
procs memory swap io system
cpu
r b w swpd free buff cache si so bi bo in cs us sy
id
0 1 1 74420 6520 30616 405128 0 0 1280 0 287 487 0 1
99
0 1 0 74420 6520 30620 405168 0 0 1196 0 271 436 0 0
100
0 1 1 74420 6520 30620 405120 0 0 1496 4 289 491 0 3
97
0 1 1 74420 6520 30620 405208 0 0 1280 0 268 466 0 0
100
1 0 1 74420 6520 30620 405208 0 0 1280 0 288 482 0 1
99
1 0 1 74420 6520 30632 405200 0 0 1416 8 277 441 1 2
97
3 1 1 74416 6520 30632 405196 4 0 1284 0 284 473 0 3
97

PostgreSQL configuration (the only changes made):
=================================================
max_connections = 1024
shared_buffers = 2800
sort_mem = 8192
vacuum_mem = 8192
effective_cache_size = 32000
System:
=======
Hardware: AMD 1.2GHz Athlon 512MB SDRAM
OS: Redhat Linux 9.0 (kernel 2.4.20-8)
FS: EXT3 with Journalling mounted with noatime, UDMA5
Disk1 (Linux): IDE 20GB 7200 RPM Western Digital
Disk2 (PostgreSQL): IDE 120GB 7200 RPM Western Digital
PostgreSQL: 7.3.4
hdparm:
=======
/dev/hda: (Linux partition)
multcount = 16 (on)
IO_support = 1 (32-bit)
unmaskirq = 1 (on)
using_dma = 1 (on)
keepsettings = 0 (off)
readonly = 0 (off)
readahead = 8 (on)
geometry = 2498/255/63, sectors = 40132503, start = 0

/dev/hdc: (PostgreSQL partition)
multcount = 16 (on)
IO_support = 1 (32-bit)
unmaskirq = 1 (on)
using_dma = 1 (on)
keepsettings = 1 (on)
readonly = 0 (off)
readahead = 8 (on)
geometry = 232581/16/63, sectors = 234441648, start = 0
Schema:
=======
CREATE TABLE table1 (
id varchar(32) DEFAULT '''' NOT NULL,
colname1 varchar(10) DEFAULT ''http'' NOT
NULL,
colname2 varchar(300) DEFAULT '''' NOT NULL,
colname3 varchar(5) DEFAULT '''' NOT
NULL,
colname4 varchar(300) DEFAULT '''' NOT
NULL,
colname5 varchar(300) DEFAULT '''' NOT NULL,
colname6 integer DEFAULT 0 NOT
NULL,
colname7 integer DEFAULT 0 NOT
NULL,
colname8 integer DEFAULT 200 NOT NULL,
colname9 varchar(10) DEFAULT '''' NOT
NULL,
colname10 varchar(10) DEFAULT '''' NOT NULL,
colname11 varchar(100) DEFAULT '''' NOT
NULL,
colname12 varchar(100) DEFAULT '''' NOT NULL,
colname13 varchar(100) DEFAULT '''' NOT NULL,
colname14 varchar(20) DEFAULT '''' NOT NULL,
colname15 integer DEFAULT 640 NOT
NULL,
colname16 integer DEFAULT 480 NOT
NULL,
colname17 integer DEFAULT 120 NOT
NULL,
colname18 integer DEFAULT 90 NOT
NULL,
colname19 timestamp DEFAULT CURRENT_TIMESTAMP NOT
NULL,
colname20 timestamp DEFAULT CURRENT_TIMESTAMP NOT
NULL,
colname21 integer DEFAULT 0 NOT NULL,
colname22 integer DEFAULT 0 NOT NULL,
colname23 timestamp DEFAULT CURRENT_TIMESTAMP NOT NULL,
colname24 integer DEFAULT 0 NOT
NULL,
colname25 integer DEFAULT 0 NOT NULL,
colname26 varchar(10) DEFAULT '''' NOT
NULL,
colname28 varchar(10) DEFAULT '''' NOT NULL,
colname29 varchar(10) DEFAULT ''jpeg'' NOT
NULL,
colname30 varchar(20) DEFAULT '''' NOT NULL,
colname31 bytea ,
PRIMARY KEY (id)
) WITHOUT OIDS

CREATE UNIQUE INDEX table1_idx_1 ON table1 (colname1, colname2, colname3,
colname4, colname5)

Sigh, :-(

Stephen

推荐答案

" Stephen" < JL ***** @ xxxxxx.com>写道:
"Stephen" <jl*****@xxxxxx.com> writes:
对于大型表上的普通VACUUM而言,将性能降低了9倍是否正常?当VACUUM运行时,我的数据库变得无法使用。从阅读
新闻组,我认为VACUUM应该只减慢10%到15%。
Is it normal for plain VACUUM on large table to degrade performance by over
9 times? My database becomes unusable when VACUUM runs. From reading
newsgroups, I thought VACUUM should only slow down by 10% to 15%.




我们听说过有关人员非常显着减速的报道

已经几乎使他们的磁盘I / O带宽饱和,然后VACUUM

将他们的系统推到了响应曲线的拐点上。除了VACUUM

和测试查询之外,你还没有说b $ b说你系统中发生的事情,但我想知道磁盘驱动器已经有多忙了。 br />

另一种可能性是你的磁盘子系统配置错误

不知何故,虽然你的hdparm输出看起来不适合我的专家眼睛。


可能你应该考虑SCSI而不是IDE磁盘驱动器。

消费级IDE驱动器似乎不太好用于服务

并行负载。例如,运行pgbench和pgbench。在一台相当新的戴尔PC(P4,某些IDE驱动器或其他)上有10个客户端线程

,我看到报告的每秒交易量减少了3倍

后台VACUUM

正在运行。对我可信赖的旧式内置式惠普服务器进行相同的测试

(使用快速的SCSI驱动器)显示只有20%的减速。我将
归因于SCSI驱动器能够更好地处理并发

请求。


问候,tom lane


---------------------------(广播结束)--------- ------------------

提示1:订阅和取消订阅命令转到 ma ******* @ postgresql.org



We have heard reports of very significant slowdowns from people who were
already nearly saturating their disk I/O bandwidth, and then VACUUM
pushed their systems over the knee of the response curve. You haven''t
said anything about what was happening in your system besides the VACUUM
and the test query, but I wonder how busy the disk drive was already.

Another possibility is that you''ve got the disk subsystem misconfigured
somehow, although your hdparm output looks okay to my not-expert eyes.

Possibly you should think about SCSI instead of IDE disk drives.
Consumer-grade IDE drives seem to be not very good about servicing
parallel loads. As an example, running "pgbench" with 10 client threads
on a fairly new Dell PC (P4, some IDE drive or other), I see about a 3x
slowdown in reported transactions-per-second when a background VACUUM
is running. The same test on my trusty old built-like-a-tank HP server
(with fast-for-its-time SCSI drives) shows only a 20% slowdown. I
attribute this to the SCSI drive being better able to handle concurrent
requests.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postgresql.org


当我运行系统时系统实际上是空闲的测试(平均负载:0.01,

0.02,0.00)。当VACUUM运行时,只需在命令行运行psql并发出2

查询,负载平均值就会快速增加(平均负载:

1.77,0.60,0.21)。我已经运行PostgreSQL超过3年了,IIRC VACUUM

一直是我的

系统上有或没有FULL属性的show-stopper(来自linux) 2.2到linux 2.4,从PostgreSQL 7.1到PostgreSQL

7.3)。唯一没有改变的是IDE驱动器。问题不在于这个系统所特有的
,我有2台具有相同配置的机器和

VACUUM大大减慢了两个数据库的速度。不幸的是,我更喜欢在谷歌便宜的IDE磁盘上实现一个系统,因为我的数据库是预计会达到太字节并且SCSI成本变得过高的



是否有其他人在Linux上使用VACUUM导致性能下降?我认为可能Linux有一个糟糕的调度程序和高IO延迟,其他操作系统(例如FreeBSD)不会受到影响。虽然,有人确实告诉我Redhat

9.0 Linux 2.4.20-8已经应用了低延迟补丁的一部分,但是我确实感觉不到改进PostgreSQL。
The system is actually idling when I ran the tests (load average: 0.01,
0.02, 0.00). When VACUUM runs, load average increases quickly (load average:
1.77, 0.60, 0.21) just by running psql on command line and issuing 2
queries. I''ve been running PostgreSQL for over 3 years now and IIRC VACUUM
has always been a show-stopper with or without the FULL attribute on my
system (from linux 2.2 to linux 2.4, and from PostgreSQL 7.1 to PostgreSQL
7.3). The only thing that didn''t change is the IDE drive. The problem is not
unique to this system, I have 2 machines with the same configuration and
VACUUM slows down both databases considerably. Unfortunately, I prefer to
implement a system on cheap IDE disks a la Google because my database is
expected to hit terabytes and SCSI cost becomes prohibitive.

Is anyone else getting performance degradation with VACUUM on Linux? I''m
thinking maybe Linux has a bad scheduler and high IO latency that other OSes
(eg. FreeBSD) do not suffer from. Although, someone did tell me that Redhat
9.0 Linux 2.4.20-8 has parts of the low latency patch applied but I
certainly don''t feel the improvement on PostgreSQL.


听起来你可能是I / O绑定的。如果您放弃5路唯一索引

进行测试,真空和并行选择如何运行?


2003年10月15日星期三,Stephen写道:
It sounds like you might be I/O bound. if you drop the 5 way unique index
for a test, how do the vacuum and parallel select run?

On Wed, 15 Oct 2003, Stephen wrote:
您好,

大型桌面上的普通VACUUM通过9次降低性能是否正常?当VACUUM运行时,我的数据库变得无法使用。从阅读
新闻组,我认为VACUUM应该只减慢10%到15%。 MySQL InnoDB等其他MVCC数据库甚至可以离散地进行VACUUM(内部运行)。它是我的Linux系统还是PostgreSQL?

数据库主要是只读的。有133,000行,每行大小约为2.5kB(主要是由于bytea列保存了二进制图像)。
长行导致系统TOAST表。 VACUUM需要5分20秒才能完成。系统空转时我反复运行以下测试:

正常运行时:
==================== tsdb =#explain analyze select * from table1 where id =
''33a4e9b6eae09634f4ff3e6fa9280f6e'';
查询计划
----------------- -------------------------------------------------- ---------
------------------------------------- ---------------
使用table1上的table1_pkey进行索引扫描(成本= 0.00..6.01行= 1宽度= 346)
(实际时间= 25.30 .. 25.31行= 1循环= 1)
索引条件:(id =''33a4e9b6eae09634f4ff3e6fa9280f6e''::字符变化)
总运行时间:25.52毫秒
(3行)
<当VACUUM运行时:
=================
tsdb =#explain analyze select * from table1 where id =
'' 336139b47b7faf09fc4d4f03680a4ce5'';
查询计划
--------------------------- -------------------------------------------------
----------------------------------------------- ---------
使用table1上的table1_pkey进行索引扫描(成本= 0.00..6.01行= 1宽度= 346)
(实际时间= 2290.07..2290.10 rows = 1 loops = 1)
索引条件:(id =''336139b47b7faf09fc4d4f03680a4ce5''::字符变化)
总运行时间:2290.22毫秒
(3行)

VACUUM输出:
==============
tsdb =#VACUUM VERBOSE table1;
信息: - 关系public.table1--
信息:Pages 5887:改为0,空0; Tup 132672:Vac 0,保持0,UnUsed
144.
总CPU 0.28s / 0.01u秒过去36.08秒。
信息: - 关联pg_toast.pg_toast_12437088--
信息:40495页:更改0,空0; Tup 197587:Vac 0,Keep 0,UnUsed
235.
总CPU 1.73秒/0.20u秒过去233.91秒。


vmstat而VACUUM'' ing:
========================
procs内存交换io系统
cpu
rbw swpd免费buff缓存si所以bi bo in cs us sy
id
0 1 1 74420 6520 30616 405128 0 0 1280 0 287 487 0 1
99
0 1 0 74420 6520 30620 405168 0 0 1196 0 271 436 0 0
100
0 1 1 74420 6520 30620 405120 0 0 1496 4 289 491 0 3
97
0 1 1 74420 6520 30620 405208 0 0 1280 0 268 466 0 0
100
1 0 1 74420 6520 30620 405208 0 0 1280 0 288 482 0 1
99
1 0 1 74420 6520 30632 405200 0 0 1416 8 277 441 1 2
97
3 1 1 74416 6520 30632 405196 4 0 1284 0 284 473 0 3 ========================= ========================
max_connections = 1024
shared_buffers = 2800
sort_mem = 8192
vacuum_mem = 8192
effective_cache_size = 32000

系统:
=======
硬件:AMD 1.2GHz Athlon 512MB SDRAM
操作系统:Redhat Linux 9.0(内核2.4.20-8)
FS:EXT3 with Journalling安装了noatime,UDMA5
Disk1(Linux):IDE 20GB 7200 RPM Western Digital
Disk2(PostgreSQL):IDE 120GB 7200 RPM Western Digital
PostgreSQL:7.3.4

hdparm:
=======
/ dev / hda :( Linux分区)
multcount = 16(on)
IO_support = 1(32位)
unmaskirq = 1(on)
using_dma = 1(on)
keepsettings = 0(off)
readonly = 0(关闭)
readahead = 8(on)
geometry = 2498/255/63,sectors = 40132503,start = 0

/ dev / hdc: (PostgreSQL分区)
multcount = 16(上)
IO_support = 1(32位)
unmaskirq = 1(上)
using_dma = 1(上)
keepsettings = 1(on)
readonly = 0(关闭)
readahead = 8(on)
geometry = 232581/16/63,sector = 234441648,start = 0

架构:
=======
CREATE TABLE table1(
id varchar(32)DEFAULT''''NOT NULL,
colname1 varchar(10)DEFAULT''http''NOT
NULL,
colname2 varchar(300)DEFAULT''''NOT NULL,
colname3 varchar(5)DEFAULT''''NOT
NULL,
colname4 varchar(300 )DEFAULT''''NOT
NULL,
colname5 varchar(300)DEFAULT''''NOT NULL,
colname6 integer DEFAULT 0 NOT
NULL ,
colname7整数DEFAULT 0 NOT
NULL,
colname8整数DEFAULT 200 NOT NULL,
colname9 varchar(10)DEFAULT''''NOT
NULL,
colname10 varchar(10)DEFAULT''''NOT NULL,
colname11 varchar(100)DEFAULT''''NOT
NULL,
colname12 varchar(100)DEFAULT'''' NOT NULL,
colname13 varchar(100)DEFAULT''''NOT NULL,
colname14 varchar(20)DEFAULT''''NOT NULL,
colname15 integer DEFAULT 640 NOT
NULL,
colname16整数DEFAULT 480 NOT
NULL,
colname17 integer DEFAULT 120 NOT
NULL,
colname18整数DEFAULT 90 NOT
NULL,
colname19 timestamp DEFAULT CURRENT_TIMESTAMP NOT
NULL,
colname20 timestamp DEFAULT CURRENT_TIMESTAMP NOT
NULL ,
colname21 integer DEFAULT 0 NOT NULL,
colname22 integer DEFAULT 0 NOT NULL,
colname23 timestamp DEFAULT CURRENT_TIMESTAMP NOT NULL,
colname24 integer DEFAULT 0 NOT
NULL,
colname25整数DEFAULT 0 NOT NULL,
colname26 varchar(10)DEFAULT''''NOT
NULL,
colname28 varchar(10)DEFAULT''''NOT NULL,
colname29 varchar(10)DEFAULT''jpeg''NOT
NULL,
colname30 varchar(20)DEFAULT''''NOT NULL,
colname31 bytea,
PRIMARY KEY(id)
)没有OIDS

创建独特的索引table1_idx_1 ON table1 (colname1,colname2,colname3,
colname4,colname5)

叹了口气,: - (

Stephen


- --------------------------(广播结束)------------------- --------
提示6:您是否搜索了我们的列表档案?

http://archives.postgresql.org
Hello,

Is it normal for plain VACUUM on large table to degrade performance by over
9 times? My database becomes unusable when VACUUM runs. From reading
newsgroups, I thought VACUUM should only slow down by 10% to 15%. Other MVCC
databases like MySQL InnoDB can even VACUUM discretely (runs internally). Is
it my Linux system or is it PostgreSQL?

The database is mostly read-only. There are 133,000 rows and each row is
about 2.5kB in size (mostly due to the bytea column holding a binary image).
The long row causes system to TOAST the table. VACUUM takes 5m20s to
complete. I repeatedly ran the following tests while system is idling:
In normal operation:
====================
tsdb=# explain analyze select * from table1 where id =
''33a4e9b6eae09634f4ff3e6fa9280f6e'';
QUERY PLAN
----------------------------------------------------------------------------
----------------------------------------------------
Index Scan using table1_pkey on table1 (cost=0.00..6.01 rows=1 width=346)
(actual time=25.30..25.31 rows=1 loops=1)
Index Cond: (id = ''33a4e9b6eae09634f4ff3e6fa9280f6e''::character varying)
Total runtime: 25.52 msec
(3 rows)

When VACUUM runs:
=================
tsdb=# explain analyze select * from table1 where id =
''336139b47b7faf09fc4d4f03680a4ce5'';
QUERY PLAN
----------------------------------------------------------------------------
--------------------------------------------------------
Index Scan using table1_pkey on table1 (cost=0.00..6.01 rows=1 width=346)
(actual time=2290.07..2290.10 rows=1 loops=1)
Index Cond: (id = ''336139b47b7faf09fc4d4f03680a4ce5''::character varying)
Total runtime: 2290.22 msec
(3 rows)
VACUUM output:
==============
tsdb=# VACUUM VERBOSE table1;
INFO: --Relation public.table1--
INFO: Pages 5887: Changed 0, Empty 0; Tup 132672: Vac 0, Keep 0, UnUsed
144.
Total CPU 0.28s/0.01u sec elapsed 36.08 sec.
INFO: --Relation pg_toast.pg_toast_12437088--
INFO: Pages 40495: Changed 0, Empty 0; Tup 197587: Vac 0, Keep 0, UnUsed
235.
Total CPU 1.73s/0.20u sec elapsed 233.91 sec.
VACUUM

vmstat while VACUUM''ing:
========================
procs memory swap io system
cpu
r b w swpd free buff cache si so bi bo in cs us sy
id
0 1 1 74420 6520 30616 405128 0 0 1280 0 287 487 0 1
99
0 1 0 74420 6520 30620 405168 0 0 1196 0 271 436 0 0
100
0 1 1 74420 6520 30620 405120 0 0 1496 4 289 491 0 3
97
0 1 1 74420 6520 30620 405208 0 0 1280 0 268 466 0 0
100
1 0 1 74420 6520 30620 405208 0 0 1280 0 288 482 0 1
99
1 0 1 74420 6520 30632 405200 0 0 1416 8 277 441 1 2
97
3 1 1 74416 6520 30632 405196 4 0 1284 0 284 473 0 3
97

PostgreSQL configuration (the only changes made):
=================================================
max_connections = 1024
shared_buffers = 2800
sort_mem = 8192
vacuum_mem = 8192
effective_cache_size = 32000
System:
=======
Hardware: AMD 1.2GHz Athlon 512MB SDRAM
OS: Redhat Linux 9.0 (kernel 2.4.20-8)
FS: EXT3 with Journalling mounted with noatime, UDMA5
Disk1 (Linux): IDE 20GB 7200 RPM Western Digital
Disk2 (PostgreSQL): IDE 120GB 7200 RPM Western Digital
PostgreSQL: 7.3.4
hdparm:
=======
/dev/hda: (Linux partition)
multcount = 16 (on)
IO_support = 1 (32-bit)
unmaskirq = 1 (on)
using_dma = 1 (on)
keepsettings = 0 (off)
readonly = 0 (off)
readahead = 8 (on)
geometry = 2498/255/63, sectors = 40132503, start = 0

/dev/hdc: (PostgreSQL partition)
multcount = 16 (on)
IO_support = 1 (32-bit)
unmaskirq = 1 (on)
using_dma = 1 (on)
keepsettings = 1 (on)
readonly = 0 (off)
readahead = 8 (on)
geometry = 232581/16/63, sectors = 234441648, start = 0
Schema:
=======
CREATE TABLE table1 (
id varchar(32) DEFAULT '''' NOT NULL,
colname1 varchar(10) DEFAULT ''http'' NOT
NULL,
colname2 varchar(300) DEFAULT '''' NOT NULL,
colname3 varchar(5) DEFAULT '''' NOT
NULL,
colname4 varchar(300) DEFAULT '''' NOT
NULL,
colname5 varchar(300) DEFAULT '''' NOT NULL,
colname6 integer DEFAULT 0 NOT
NULL,
colname7 integer DEFAULT 0 NOT
NULL,
colname8 integer DEFAULT 200 NOT NULL,
colname9 varchar(10) DEFAULT '''' NOT
NULL,
colname10 varchar(10) DEFAULT '''' NOT NULL,
colname11 varchar(100) DEFAULT '''' NOT
NULL,
colname12 varchar(100) DEFAULT '''' NOT NULL,
colname13 varchar(100) DEFAULT '''' NOT NULL,
colname14 varchar(20) DEFAULT '''' NOT NULL,
colname15 integer DEFAULT 640 NOT
NULL,
colname16 integer DEFAULT 480 NOT
NULL,
colname17 integer DEFAULT 120 NOT
NULL,
colname18 integer DEFAULT 90 NOT
NULL,
colname19 timestamp DEFAULT CURRENT_TIMESTAMP NOT
NULL,
colname20 timestamp DEFAULT CURRENT_TIMESTAMP NOT
NULL,
colname21 integer DEFAULT 0 NOT NULL,
colname22 integer DEFAULT 0 NOT NULL,
colname23 timestamp DEFAULT CURRENT_TIMESTAMP NOT NULL,
colname24 integer DEFAULT 0 NOT
NULL,
colname25 integer DEFAULT 0 NOT NULL,
colname26 varchar(10) DEFAULT '''' NOT
NULL,
colname28 varchar(10) DEFAULT '''' NOT NULL,
colname29 varchar(10) DEFAULT ''jpeg'' NOT
NULL,
colname30 varchar(20) DEFAULT '''' NOT NULL,
colname31 bytea ,
PRIMARY KEY (id)
) WITHOUT OIDS

CREATE UNIQUE INDEX table1_idx_1 ON table1 (colname1, colname2, colname3,
colname4, colname5)

Sigh, :-(

Stephen


---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org



------------------- --------(广播结束)---------------------------

提示4 :不要杀了-9''邮政局长


---------------------------(end of broadcast)---------------------------
TIP 4: Don''t ''kill -9'' the postmaster


这篇关于VACUUM显着降低了性能。数据库变得无法使用!的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆