流处理很多的东西OVA [英] Stream processing lots of stuff to OVA

问题描述

所以我们的开发商需要我批了一堆资料和加工成的OVA是psented回来下载$ P $。这是使用长法（即写入文件系统）一个简单的过程，但开发商想要一个更清洁，精简的解决方案，将更好的规模。因此，他们要求我流这被证明是困难的整个过程。是否有人可以给我一些指导。这里是需要完成以下步骤：

1. 从网络服务器获取输入（Web服务器将最终通过这些作为流。）
   
   
   
   • 随机密码
   
   
   • XML文件
   
   


2. 修改启动脚本文件系统（即插入由服务器生成随机密码）


3. XML文件和引导脚本创建ISO


4. 计算ISO的SHA1总和


5. ISO的SHA1追加一笔清单文件OVF目录


6. 从OVF目录中创建OVA

下面是一个例子的目录结构（Ⅰ在/只是为了简单概述此）

  /  - 
  |
  | --iso /
  | |
  | | --boot.sh（如随机密码被插入）
  | | --config.xml（这是从Web服务器递给需要从服务器流式传输）。
  |
  | --OVF /
      |
      | --disk.vmdk
      | --ovf.xml
      | --manifest.mf（包含在OVF目录中的所有文件的SHA1）
      | --boot.iso（此文件将存在一次从ISO目录中创建）
 

下面是我到目前为止（我会再具体解释问题是...有很多的问题。）：

 猫/ISO/boot.sh | SEDS％DEFAULT％RANDOM％| mkisofs的/ ISO / * |回声SHA1（boot.iso的）=＆放大器;＆安培; SHA1SUM＆GT;＆GT; MANIFEST.MF |焦油-cvf success.ova / OVF / *
 

的 注意的
在boot.sh有一个变量设置为默认这样的（只是用于测试）：

  PASSWORD =DEFAULT
 

的 注意的
这是在manifest文件中的一行应该是什么样子：

  SHA1（boot.iso的）= 5fbc0d70等等等等a91c9121bb
 

所以，我从来没有尝试之前在一个流写一个完整的脚本。通常我写的文件系统有很多，我去。第一个问题，我这个看到的是，SED是替换字符串，但它的管道到mkisofs的将不能作为mkiosfs只是要做出什么发现在/ ISO的ISO。我甚至不知道你是否可以通过类似的东西，以mkisofs的。管道有时是奇怪的思考。

接下来，我想mkisofs的是好的，因为我没有指定一个文件输出，因此应输出到标准输出将被传递给SHA1SUM，但这里是下一个问题，我看到。我需要一些额外的文本追加到该文件之前，SHA1总和被添加其中还挺中断数据流。

最后，最后的问题，我看到的是如何通过一切是焦油成OVA无需编写到文件系统中第一个（写入MANIFEST.MF）。

哦，和我应该首先提到的最后一个大问题是config.xml文件。现在即时处理它只是一个文件。 dev的家伙想将它传递给这个脚本作为流为好。我没有线索如何处理这个问题。

任何帮助将大大AP preciated。这些概念是有点超出我的知识。

谢谢！

更新13年12月11日下午2点11 EST 的

单独现在测试每个部分。下面很快将报告结果。

更新13年12月11日下午2:14 EST 的

以下工作：

 猫/ISO/boot.sh | SEDS％DEFAULT％RANDOM％
 

和输出如下：

  RANDOM =RANDOM
 

完全符合市场预期。

您是正确的NeronLeVelu，我会稍后回来看看更仔细地被实时生成随机密码时sed的。 IE浏览器。确保正确的字符转义。现在虽然，我只是测试的逻辑。我会担心的正则表达式，后来逃跑。我们甚至还没有决定随机密码呢。这只是暂时的，并且很可能是字母数字。

移动到下一个部分。仍不能确定如何采取从sed的（标准输出）输出，并用它在ISO创建到包括但实际创建了被写入到文件系统中的文件。没有写入文件系统，可能是不可能的。更多来尽快

解决方案

 ＃输入密码，如果它包含＆安培; \\，并在用分离器的的sed（默认为/）
Password4Sed =`回声\\$ {PASSWORD} | SED \\S / [\\\\ /＆安培] / \\\\\\\\＆安培/克\\`
＃没有必要的猫与SED
SEDS / DEFAULT / $ {} Password4Sed // ISO / boot.sh＆GT; /tmp/mkisofs.input
 

从此输入善待休息和放些测试来验证像空的CRC值或mkisofs.input每一个步骤。这将有助于在运行时错误生产时发生

So one of our developers needs me to batch a bunch of information and process it into an OVA to be presented back for download. This is an easy process using the long method (ie writing to the filesystem), but the developers want a cleaner, streamlined solution that will scale better. They have therefore requested that I stream the entire processes which is proving difficult. Can someone please give me some direction. Here are the steps that need to be accomplished:

1. Get input from webserver (Webserver will pass these as stream eventually.)
   • Random password
   • XML file
2. Modify boot script on file system (ie insert random password generated by server)
3. Create ISO of XML file and boot script
4. Calculate the SHA1 sum of ISO
5. Append SHA1 sum of ISO to manifest file in OVF directory
6. Create OVA from OVF directory

Here is an example directory structure (I outlined this in / just for simplicity)

/--
  |
  |--ISO/
  |   |
  |   |--boot.sh (Where the random password gets inserted)
  |   |--config.xml (This is handed from the web server. Needs to stream from server)
  |
  |--OVF/
      |
      |--disk.vmdk
      |--ovf.xml
      |--manifest.mf (Contains SHA1 of all files in OVF directory)
      |--boot.iso (This file will exist once created from ISO directory)

Here is what I have so far (I'll explain the issues afterwards. Yes... there are a lot of issues):

cat /ISO/boot.sh | sed "s%DEFAULT%RANDOM%" | mkisofs /ISO/* | echo "SHA1(boot.iso)= " && sha1sum >> manifest.mf | tar -cvf success.ova /OVF/*

NOTE In boot.sh there is a variable set to DEFAULT like this (Just for testing purposes):

PASSWORD="DEFAULT"

NOTE This is what a line in the manifest file should look like:

SHA1(boot.iso)= 5fbc0d70 BLAH BLAH BLAH a91c9121bb

So I've never tried to write an entire script in one stream before. Usually I write to the filesystem a lot as I go. The first issue I see with this is that sed is replacing the string, but what it's piping over to mkisofs will not be used as mkiosfs is just going to make an iso of what it finds in /ISO. I dont even know if you can pass something like that to mkisofs. Piping is sometimes weird to think about.

Next, I think mkisofs is ok because I didnt specify a file output, therefore it should output to stdout which will be passed to sha1sum, but and here is the next problem I see. I need to append some additional text to the file before the SHA1 sum gets added which kinda interrupts the stream.

Finally, the last problem I see is how to pass everything to be tar into OVA without writing to the filesystem first (writing to manifest.mf).

Oh and the last BIG problem which I should have mentioned first is the config.xml file. Right now im dealing with it as just a file. The dev guys want to pass it to this script as a stream as well. I dont have a clue how to handle that.

Any help would be greatly appreciated. These concepts are a little beyond my knowledge.

Thanks!

UPDATE 12/11/13 2:11PM EST

Testing each part individually right now. Will report findings below soon.

UPDATE 12/11/13 2:14PM EST

The following works:

cat /ISO/boot.sh | sed "s%DEFAULT%RANDOM%"

and produces the following output:

RANDOM="RANDOM"

Exactly as expected.

You are correct NeronLeVelu, I will have to come back later and look at sed more carefully when real random passwords are being generated. ie. Making sure proper characters are escaped. Right now though, I'm just testing the logic. I will worry about regex and escaping later. We have not even decided on random password yet. It's only temporary and will most likely be alphanumeric.

Moving onto next part. Still not sure how to take the output from sed (stdout) and use it to include in ISO creation without actually creating a file that gets written to the file system. It may not be possible without writing to file system. More to come soon

解决方案

# for the password if it contain & \ and separator used in your sed (default is /)
Password4Sed="`echo \"${PASSWORD} | sed \"s/[\\/&]/\\\\&/g\"`"


# no need of a cat with a sed
sed "s/DEFAULT/${Password4Sed}/"/ISO/boot.sh > /tmp/mkisofs.input

Treat rest from this input and put some test to validate each step like empty crc value or mkisofs.input. This will help at runtime when production error occur

这篇关于流处理很多的东西OVA的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！