使用make和Makefile从包含空格的文件路径中处理原始源数据 [英] Working from raw source data from a filepath containing spaces using make and Makefiles

问题描述

我有一个使用python脚本和Makefile的存储库.我想要一个设置程序使他们能够轻松设置环境并从我们的服务器复制必要的数据文件.

I have a repository that uses python scripts and a Makefile. I want to have a setup procedure that allows them to easily set up an environment and copy in the necessary data files from our server.

在Makefile中包含源数据文件的问题是公司服务器使用驱动器名称中的空格，这个空格不太喜欢，因此我可以将这些文件作为依赖项列出用于目标输出文件.

The problem with including the source data files in the Makefile is that the company server uses spaces in the drive name, which make doesn't like very much, so I can list those files as dependencies for the target output file.

我当前的Makefile基本上仅执行以下操作:

My current Makefile basically does only the following:

.PHONY : all
all : output.csv

.PHONY : copy_data_to_local_folder
copy_data_to_local_folder :
    python copyfile.py "V:\\Server Path\\With Spaces\\Inputs 1.csv" local/inputs1.csv
    python copyfile.py "V:\\Server Path\\With Spaces\\Inputs 2.csv" local/inputs2.csv

output.csv : combine_data.R local/inputs1.csv local/inputs2.csv
    Rscript $^ $@

copy_data_to_local_folder 部分仅用于将数据获取到本地目录，但不包括在内在DAG中导致生成 output.csv (即 all:output.csv copy_data_to_local_folder )目标将需要每次运行.

The copy_data_to_local_folder part is just to get the data to the local directory, but it isn't included in the DAG leading to the production of output.csv (i.e. all : output.csv copy_data_to_local_folder) or else the target would need to run everytime.

以下是我的解决方案，但是我不确定什么是最佳实践:

My solution ideas are the following, but I'm not sure what's best practice:

1. 使用其他制作工具.我可以在Python中使用 Luigi 或在R中使用 Drake ，但我希望保留该工具更具通用性.

1. Use a different make tool. I could use Luigi in Python or Drake in R, but I would prefer to keep the tool somewhat more generalized.

运行安装脚本以复制文件.我认为将有一种方法可以运行文件复制脚本作为环境设置的一部分，但是我不熟悉该怎么做.

Run a setup script to copy in files. I assume there would be a way to run the file copying scripts as part of the environment setup, but I am unfamiliar with how to do this.

我不确定执行此操作的最佳方法.我希望能够与同事共享代码并让他们能够在他们的系统上启动并运行，而无需进行过多的配置.有没有最好的这种情况下练习吗?

I am not sure about the best way to do this. I want to be able to share the code with a co-worker and have them be able to get up and running on their system without too much messing around to configure. Is there a best practice for this situation?

推荐答案

一个解决方法是:

local/inputs1.csv :
    python copyfile.py "V:\\Server Path\\With Spaces\\Inputs 1.csv" $@
local/inputs2.csv :
    python copyfile.py "V:\\Server Path\\With Spaces\\Inputs 2.csv" $@

output.csv : combine_data.R | local/inputs1.csv local/inputs2.csv
    Rscript $^ $| $@

请注意，将 local/inputs1.csv 和 local/inputs2.csv 设置为仅订购的先决条件，因此只有在它们不存在时才进行制造(除非您想在每次运行makefile时复制它们).自动变量 $ | 表示仅订购的先决条件，它们不包含在 $ ^ 中.

Note that local/inputs1.csv and local/inputs2.csv are made order-only prerequisites, so that they are only made when they don't exist (unless you'd like to copy them every time the makefile is run). Automatic variable $| refers to order-only prerequisites, they aren't included in $^.

这篇关于使用make和Makefile从包含空格的文件路径中处理原始源数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！