Ansible: copy a directory recursive - take two

Posted by ads' corner on Thursday, 2019-11-28
Posted in [Ansible][Linux]

Another slow attempt at copying directories and files on a remote system from one place to another, using just Ansible operations.

This time I’m not using with_filetree, but building the tree beforehand, using the find module:

1
2
3
4
5
6
- name: Find entries
  find:
    path: /path/to/source/directory
    recurse: yes
    file_type: any
  register: source_tree

Afterwards the operation is split into two parts. First create all directories:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
- name: Create the directories
  file:
    path: "{{ item.path | regex_replace('/path/to/source/directory/','/path/to/destination/directory/') }}"
    state: directory
    mode: "{{ item.mode }}"
  loop: "{{ source_tree.files | flatten(levels=1) }}"
  loop_control:
    label: "{{ item.path | regex_replace('/path/to/source/directory', '') }}"
  when:
    - item.isdir

The performance of this task is not too bad, but not fast either. The second step is copying all the files:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
- name: Copy the files
  copy:
    src: "{{ item.path }}"
    dest: "{{ item.path | regex_replace('/path/to/source/directory/','/path/to/destination/directory/') }}"
    remote_src: yes
    mode: "{{ item.mode }}"
  loop: "{{ source_tree.files | flatten(levels=1) }}"
  loop_control:
    label: "{{ item.path | regex_replace('/path/to/source/directory', '') }}"
  when:
    - item.isdir == False

This task is really slow, needs between 1-2 seconds per file. Just documenting this here, and probably moving on to call rsync directly.


Categories: [Ansible] [Linux]