同时多个请求到非常简单的数据库 [英] simultaneous multiple requests to very simple database
问题描述
我有一个应用程序，我需要一个非常简单的数据库，实际上是一个非常大的字典。必须同时从
多个进程访问非常大的字典。当记录被写入时，我需要能够在非常大的字典中锁定记录
。估计
记录的数量将在他早期的b
$和
早期阶段达到50,000到100,000。每条记录大约要花费100到150美元。
速度不是一个大问题，虽然我必须完成少量处理
超过90秒。延迟时间越长，但是为了保持吞吐量，必须并行运行更多的b / b
流程。
这是我们通常的权衡取舍来了解和爱。
尽管我还有另一个项目，但字典不必持续超过
父流程的生命周期这个
会是一个好主意。
此时此刻，我知道他们会是一些善意的灵魂暗示各种各样的
SQL解决方案。虽然我很欣赏这个想法，但不幸的是我没有时间来解决另一个组件问题。总有一天我会搞清楚
因为我真的很喜欢我用SQL lite看到的东西但不幸的是，今天
不是那天（除非他们会给我他们的工作，回家和小区
电话号码所以我可以在我被卡住时打电话。;-)
所以想到的解决方案是共享的某种形式的字典
带锁定信号量记分板或多线程进程的内存
包含单个数据库（Python原生字典，metakit，gdbm ??）
并且有我的所有进程都使用xmlrpc对它说话，这给我带来了如何用xmlrpc创建多线程服务器的问题。
所以反馈和指针信息将是最受欢迎的。我是
还在探索这个想法所以我对所有建议都持开放态度（除了
也许是SQL :-)
--- eric
" Eric S. Johansson" < es*@harvee.org>在留言中写道
新闻：ma ************************************ ** @ pyth on.org ...
< snip>此时，我知道他们会成为某种善意的人提出各种各样的解决方案。虽然我很欣赏这个想法，但遗憾的是我还没有时间来解决另一个问题。总有一天我会弄清楚
因为我真的很喜欢我用SQL lite看到的东西但不幸的是，今天
不是那一天（除非他们会给我他们的工作，家庭和手机电话号码）所以当我被卡住时我可以打电话。; - ）
< snip>
如果这个回复听起来有点油腻，请原谅我。但我的意思是没有恶意。
你是否真的希望编写自己的（数据库）解决方案，这个
将节省你的时间和精力学习现有的（SQL）解决方案？
因为 -
如果你想节省时间在谜题中，你肯定会以错误的方式回答
。
祝你好运
Thomas Bartkus
Thomas Bartkus写道：" Eric S. Johansson" < es*@harvee.org>在消息中写道
新闻：ma ************************************** @ pyth on.org ...
< snip>
此时，我知道他们会成为某种善意的灵魂，建议各种各样的SQL解决方案。虽然我很欣赏这个想法，但遗憾的是我还没有时间来解决另一个问题。总有一天我会弄清楚
因为我真的很喜欢我用SQL lite看到的东西但不幸的是，今天
不是那一天（除非他们会给我他们的工作，家庭和手机电话号码）所以当我被卡住时我可以打电话。;-)
< snip>
如果这个回复听起来有点油腻，请原谅我。但我的意思是没有恶意。
理解并采取这种精神。
你真的希望自己编写（数据库）解决方案，这将节省您学习现有（SQL）解决方案的时间和精力吗？
因为 -
如果您正在寻求节省时间在谜题中，你肯定会以错误的方式对待它。
很久以前我学到的一件事就是尊重唠叨的声音在
我脑子里写着有什么不对劲。现在
数据库，这个声音不是唠叨而是尖叫。所以我做了我的
查询来试图证明直觉是错误的。到目前为止，这还没有发生。
当我查看数据库时，我看到一堆非常好的解决方案，无论是过于复杂还是重量级的一方面非常好并且简单但无法处理并发性的另一方面。两套
点解决方案，试图让自己和开发人员适应其他应用程序环境。
99.9％我做了什么（我怀疑这可能适用于其他人）
可以通过略微增强的超级词典来满足，并且记录
级锁定。但是，数据库世界不适合这种模式。它有更多的复杂功能，然后经常是必要的。
如果我找到时间，我会尝试建立这样一个野兽可能
。不情愿的唯一原因是我多花了几个小时跟踪操作系统级别的并发问题多年以前的b $ b，所以我不创建多线程应用程序很简单。
总之，我查询的唯一原因是看看我是否缺少一个解决方案。到目前为止，我还没有找到任何使用的工作因为
它们比带有文件
锁定的简单dbm增加了更复杂的数量级。显然，这个简单的解决方案性能非常糟糕
现在我需要简单的实现。
感谢您的评论。
--- eric
2005年1月18日星期二17:33:26 -0500，Eric S. Johansson写道：
< blockquote class =post_quotes>当我查看数据库时，我看到一堆非常好的解决方案，一方面过于复杂或重量级，而且非常简单，但无法处理并发性在另一。两套解决方案，试图让自己和开发人员适应其他应用程序环境。
你考虑过SQLite / pySQLite吗？
-
里卡多
I have an application where I need a very simple database, effectively a
very large dictionary. The very large dictionary must be accessed from
multiple processes simultaneously. I need to be able to lock records
within the very large dictionary when records are written to. Estimated
number of records will be in the ballpark of 50,000 to 100,000 in his
early phase and 10 times that in the future. Each record will run about
100 to 150 bytes.
speed is not a huge concern although I must complete processing in less
than 90 seconds. The longer the delay however the greater number of
processes must be running parallel in order to keep the throughput up.
It''s the usual trade-off we have all come to know and love.
it is not necessary for the dictionary to persist beyond the life of the
parent process although I have another project coming up in which this
would be a good idea.
at this point, I know they will be some kind souls suggesting various
SQL solutions. While I appreciate the idea, unfortunately I do not have
time to puzzle out yet another component. Someday I will figure it out
because I really liked what I see with SQL lite but unfortunately, today
is not that day (unless they will give me their work, home and cell
phone numbers so I can call when I am stuck. ;-)
So the solutions that come to mind are some form of dictionary in shared
memory with locking semaphore scoreboard or a multithreaded process
containing a single database (Python native dictionary, metakit, gdbm??)
and have all of my processes speak to it using xmlrpc which leaves me
with the question of how to make a multithreaded server using stock xmlrpc.
so feedback and pointers to information would be most welcome. I''m
still exploring the idea so I am open to any and all suggestions (except
maybe SQL :-)
---eric
"Eric S. Johansson" <es*@harvee.org> wrote in message
news:ma**************************************@pyth on.org...
<snip>at this point, I know they will be some kind souls suggesting various
SQL solutions. While I appreciate the idea, unfortunately I do not have
time to puzzle out yet another component. Someday I will figure it out
because I really liked what I see with SQL lite but unfortunately, today
is not that day (unless they will give me their work, home and cell
phone numbers so I can call when I am stuck. ;-)
<snip>
Forgive me if this reply sounds a bit glib. But I do mean it without malice.
Do you seriously expect to write your own (database) solution and that this
will save you time and effort over learning an existing (SQL) solution?
Because -
If you are seeking to "save time" on "puzzles", you are certainly going
about it the wrong way.
Best of luck
Thomas Bartkus
Thomas Bartkus wrote:"Eric S. Johansson" <es*@harvee.org> wrote in message
news:ma**************************************@pyth on.org...
<snip>at this point, I know they will be some kind souls suggesting various
SQL solutions. While I appreciate the idea, unfortunately I do not have
time to puzzle out yet another component. Someday I will figure it out
because I really liked what I see with SQL lite but unfortunately, today
is not that day (unless they will give me their work, home and cell
phone numbers so I can call when I am stuck. ;-)
<snip>
Forgive me if this reply sounds a bit glib. But I do mean it without malice.
understood and taken in that spirit.
Do you seriously expect to write your own (database) solution and that this
will save you time and effort over learning an existing (SQL) solution?
Because -
If you are seeking to "save time" on "puzzles", you are certainly going
about it the wrong way.
one thing I learned a long time ago was to respect the nagging voice in
the back of my head that says "there is something wrong". Right now
with databases, that voice is not nagging but screaming. So I made my
query to try and prove that intuition wrong. So far, that has not happened.
When I look at databases, I see a bunch of very good solutions that are
either overly complex or heavyweight on one hand and very nice and
simple but unable to deal with concurrency on the other. two sets of
point solutions that try to stretch themselves and the developers to fit
other application contexts.
99.9 percent of what I do (and I suspect this could be true for others)
could be satisfied by a slightly enhanced super dictionary with a record
level locking. but, the database world does not fit this model. It has
a great deal more complication then what is frequently necessary.
If I ever find the time, I will try to build such a beast probably
around Metakit. The only reason for reluctance is that I have spent too
many hours tracking down concurrency problems at the OS level way to
many years ago and so I do not create multithreaded applications lightly.
so in conclusion, my only reason for querying was to see if I was
missing a solution. So far, I have not found any work using because
they add orders of magnitude more complexity than simple dbm with file
locking. Obviously, the simple solution has horrible performance right
now I need simplicity implementation.
thanks for your commentary.
---eric
On Tue, 18 Jan 2005 17:33:26 -0500, Eric S. Johansson wrote:
When I look at databases, I see a bunch of very good solutions that are
either overly complex or heavyweight on one hand and very nice and simple
but unable to deal with concurrency on the other. two sets of point
solutions that try to stretch themselves and the developers to fit other
application contexts.
Have you considerded SQLite/pySQLite ?
--
Ricardo
这篇关于同时多个请求到非常简单的数据库的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
