Skip to content

Question about deployment CAPE on Azure environment #1156

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
6 tasks done
rleal124 opened this issue Sep 21, 2022 · 62 comments
Closed
6 tasks done

Question about deployment CAPE on Azure environment #1156

rleal124 opened this issue Sep 21, 2022 · 62 comments

Comments

@rleal124
Copy link

Question about deployment CAPE on Azure environment

This is opensource and you getting free support so be friendly!

  • Free support from doomedraven ended, no whiskey no support. For something he updated the documentation :)

Prerequisites

Please answer the following questions for yourself before submitting an issue.

  • I am running the latest version
  • I did read the README!
  • I checked the documentation and found no answer
  • I checked to make sure that this issue has not already been filed
  • I'm reporting the issue to the correct repository (for multi-repository projects)
  • I'm have read all configs with all optional parts

Expected Behavior

Install the CAPE on Azure environment. Understand what reequipments addcional I need to meet

Current Behavior

I am read the documentation present https://capev2.readthedocs.io/en/latest/installation/host/cloud.html.
And also I am read and try understand the configurations presents on config path in specific the az.conf
And for last but not less import I review the cape2.sh
But I still have some questions.

  1. For Azure environment is still necessary install the libvirt and KVM, before?
  2. Any advice for Azure installation?

Failure Information (for bugs)

Please help provide information about the failure if this is a bug. If it is not a bug, please remove the rest of this template.

@github-actions
Copy link

@rleal124: hello! 👋

This issue is being automatically closed because it does not follow the issue template.

This is open source project!
So please apreciate our time that we sacrify from other thing that we could enjoy, instead of asking boring things over and over.

@doomedraven
Copy link
Collaborator

@cccs-kevin can you answer please his question?

@doomedraven
Copy link
Collaborator

but it doesn't look like you need any of those libraries, you just need azure libs https://github.com/kevoreilly/CAPEv2/blob/master/modules/machinery/az.py

@rleal124
Copy link
Author

First thanks @doomedraven for your feedback.
I can remove/comment from cape2.sh the part for libvirt ?
And this line pip install azure-identity msrest msrestazure azure-mgmt-compute azure-mgmt-network ?

@doomedraven
Copy link
Collaborator

yes, the script is mine so it's done for me and my needs, so everyone can optimise for their needs ;)

@rleal124
Copy link
Author

Thanks.

@rleal124
Copy link
Author

About the interface variable NETWORK_IFACE. I think is interface will connect VM for analysis. Correct ?
On Azure environment, the NETWORK_IFACE should connect to RG3?

@doomedraven
Copy link
Collaborator

doomedraven commented Sep 22, 2022 via email

@rleal124
Copy link
Author

rleal124 commented Sep 22, 2022

Thanks for you patience and help.
So, the following variables is for TOR, correct ?

NETWORK_IFACE=virbr1
# for tor
IFACE_IP="192.168.1.1"

@cccs-kevin
Copy link
Collaborator

Hey @rleal124, for your first question (which @doomedraven nailed), you do not need libvirt and KVM since we are interacting with the Azure platform via web calls. You do need the following packages though.

For you second question, here is the documentation that I've written to assist with an Azure + CAPE deployment:

(Also I'll be reverting the most recent PR I made to the Azure machinery after testing it in production, so stay tuned)

@cccs-kevin
Copy link
Collaborator

About the interface variable NETWORK_IFACE. I think is interface will connect VM for analysis. Correct ? On Azure environment, the NETWORK_IFACE should connect to RG3?

In the Azure environment, the NETWORK_IFACE variable in the cape.sh script should be set to the name of the NIC for VNET2_SUB2_NIC.

@rleal124
Copy link
Author

Hi @cccs-kevin, Thanks a lot your comments and answers.
I will modify the cape2.sh and remove libvirt install and add the following lines:

sudo -u ${USER} bash -c 'export PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring; poetry run python -m pip install azure-identity msrest msrestazure azure-mgmt-compute azure-mgmt-network'
#sudo usermod -aG kvm ${USER}
#sudo usermod -aG libvirt ${USER}

Also identify the mongoDB 6 is not supported on Ubuntu 22.04.

@doomedraven
Copy link
Collaborator

i guess we can start adding ifs for machinery in that script

@doomedraven
Copy link
Collaborator

about mongo is weird as i can see it here https://repo.mongodb.org/apt/ubuntu/dists/jammy/mongodb-org/6.0/

@rleal124
Copy link
Author

From mongoDB documentation:

image

@doomedraven
Copy link
Collaborator

well docs not always being updated, as sources always speak the truth and not docs ;)

@rleal124
Copy link
Author

Thanks @doomedraven, good to know. Today I am learned a new stuff.

@doomedraven
Copy link
Collaborator

after many years dealing with open source and software in general, you will learn to trust more to code than any docs :D #SadTruth

@cccs-kevin
Copy link
Collaborator

Hey @rleal124, for your first question (which @doomedraven nailed), you do not need libvirt and KVM since we are interacting with the Azure platform via web calls. You do need the following packages though.

For you second question, here is the documentation that I've written to assist with an Azure + CAPE deployment:

* https://capev2.readthedocs.io/en/latest/installation/host/cloud.html

* https://capev2.readthedocs.io/en/latest/installation/guest/saving.html#azure

* If you have any questions or something is unclear, let me know!

(Also I'll be reverting the most recent PR I made to the Azure machinery after testing it in production, so stay tuned)

Here is the PR for the updated Azure machinery #1164

@rleal124
Copy link
Author

Thanks a lot for your help.

I will starting the deployment, by first implement the RG.

I will keep posted about he progress.

Again many thanks

@doomedraven
Copy link
Collaborator

just be careful, old good meme xD
image

@rleal124
Copy link
Author

Hello again.

I am install the cape using the script, and replace the az.py for #1164.

I got the following error.

Sep 23 07:33:20 capesbxhost01 python3[30030]: 2022-09-23 07:33:20,794 [modules.machinery.az] WARNING: Failed to <bound method GalleryImagesOperations.get of <azure.mgmt.compute.v2019_12_01.operations._gallery_images_operations.GalleryImagesOperations object at 0x7ff8ce087fd0>>(('it-cs_CAPESandbox_resources', 'CAPESandbox_Gallery', 'CAPESandbox_Image')) due to the Azure error ''ClientSecretCredential' object has no attribute 'signed_session'': 'AttributeError("'ClientSecretCredential' object has no attribute 'signed_session'")'. Sep 23 07:33:20 capesbxhost01 python3[30030]: 2022-09-23 07:33:20,794 [root] CRITICAL: CuckooCriticalError: Gallery image 'CAPESandbox_Image' does not exist

But if I using the old az.py I am not got error above.

Any advice?

@cccs-kevin
Copy link
Collaborator

Try running the following from the /opt/CAPEv2 directory : poetry run python -m pip install -U azure-identity msrest msrestazure azure-mgmt-compute azure-mgmt-network

@rleal124
Copy link
Author

Not take any effect. I edited the cape2.sh and add the following line, for install the azure modules during the installation
sudo -u ${USER} bash -c 'export PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring; poetry run python -m pip install azure-identity msrest msrestazure azure-mgmt-compute azure-mgmt-network'

@cccs-kevin
Copy link
Collaborator

Can you show me the versions of those packages via poetry run python -m pip freeze ? And just to confirm, you are using the most up-to-date version of az.py?

@rleal124
Copy link
Author

About the MongoDB
I need use the mongo for Focal. If I used the Jammy, when I run apt search mongodb-org

sudo apt search mongodb-org
Sorting... Done
Full Text Search... Done

The solution I found is used for Focal:

echo "deb [signed-by=/etc/apt/keyrings/mongo.gpg arch=amd64] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/6.0 multiverse" > /etc/apt/sources.list.d/mongodb.list
curl -LO http://archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.16_amd64.deb
sudo dpkg -i ./libssl1.1_1.1.1f-1ubuntu2.16_amd64.deb

@rleal124
Copy link
Author

Can you show me the versions of those packages via poetry run python -m pip freeze ? And just to confirm, you are using the most up-to-date version of az.py?

I am install form scratch and I confirm I using the last a az.py. I don't the previous error now.
Any way in below the output from poetry run python -m pip freeze

poetry run python -m pip freeze
(...)
azure-common==1.1.28
azure-core==1.25.1
azure-identity==1.11.0
azure-mgmt-compute==28.0.0
azure-mgmt-core==1.3.2
azure-mgmt-network==21.0.1
(...)

I am just running the first analysis for test.
After doing a clean installation of cape, we have this behavior where after the timeout of 5 minutes, the machine is instantiated. As soon as we send an analysis, the machine that would theoretically be ready to use, is not recognized and ends up being re-instantiated in a loop.

Any ideas about this behavior?
In below the cape.service log, (I add -d flag for debug)

Sep 23 12:34:57 capesbxhost02 python3[2799]: 2022-09-23 12:34:57,121 [lib.cuckoo.core.scheduler] INFO: Using "az" machine manager with max_analysis_count=0, max_machines_count=10, and max_vmstartup_count=5
Sep 23 12:34:57 capesbxhost02 python3[2799]: 2022-09-23 12:34:57,127 [modules.machinery.az] DEBUG: Connecting to Azure for the region 'WestEurope'.
Sep 23 12:34:57 capesbxhost02 python3[2799]: 2022-09-23 12:34:57,133 [modules.machinery.az] DEBUG: Trying <bound method GalleryImagesOperations.get of <azure.mgmt.compute.v2022_01_03.operations._gallery_images_operations.GalleryImagesOperations object at 0x7f8894cbf610>>(('it-cs_CAPESandbox_resources', 'CAPESandbox_Gallery', 'CAPESandbox_Image'))
Sep 23 12:34:57 capesbxhost02 python3[2799]: 2022-09-23 12:34:57,652 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetsOperations.list of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_sets_operations.VirtualMachineScaleSetsOperations object at 0x7f8894cbf7f0>>(('it-cs_CAPESandbox_resources',))
Sep 23 12:34:57 capesbxhost02 python3[2799]: 2022-09-23 12:34:57,857 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetsOperations.begin_update of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_sets_operations.VirtualMachineScaleSetsOperations object at 0x7f88955bbaf0>>(('it-cs_CAPESandbox_resources', 'CAPEAnalysisVMSS', <azure.mgmt.compute.v2022_08_01.models._models_py3.VirtualMachineScaleSet object at 0x7f88955bba00>))
Sep 23 12:35:59 capesbxhost02 python3[2799]: 2022-09-23 12:35:59,901 [modules.machinery.az] DEBUG: Trying <bound method SubnetsOperations.get of <azure.mgmt.network.v2022_01_01.operations._operations.SubnetsOperations object at 0x7f88953049d0>>(('it-cs_CAPESandbox', 'vnet2_CAPESandbox', 'VNET2_SUB2'))
Sep 23 12:36:00 capesbxhost02 python3[2799]: 2022-09-23 12:36:00,142 [modules.machinery.az] DEBUG: Adding machines to database for CAPEAnalysisVMSS.
Sep 23 12:36:00 capesbxhost02 python3[2799]: 2022-09-23 12:36:00,149 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetVMsOperations.list of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_set_vms_operations.VirtualMachineScaleSetVMsOperations object at 0x7f88955b8fd0>>(('it-cs_CAPESandbox_resources', 'CAPEAnalysisVMSS'))
Sep 23 12:36:00 capesbxhost02 python3[2799]: 2022-09-23 12:36:00,150 [modules.machinery.az] DEBUG: Trying <bound method NetworkInterfacesOperations.list_virtual_machine_scale_set_network_interfaces of <azure.mgmt.network.v2022_01_01.operations._operations.NetworkInterfacesOperations object at 0x7f8896378790>>(('it-cs_CAPESandbox_resources', 'CAPEAnalysisVMSS'))
Sep 23 12:36:00 capesbxhost02 python3[2799]: 2022-09-23 12:36:00,332 [modules.machinery.az] DEBUG: CAPEAnalysisVMSS_1: Initializing...
(..)
Sep 23 12:40:50 capesbxhost02 python3[2799]: 2022-09-23 12:40:50,627 [modules.machinery.az] DEBUG: CAPEAnalysisVMSS_1: Initializing...
Sep 23 12:41:00 capesbxhost02 python3[2799]: Exception in thread Thread-5 (_thr_wait_for_ready_machine):
Sep 23 12:41:00 capesbxhost02 python3[2799]: Traceback (most recent call last):
Sep 23 12:41:00 capesbxhost02 python3[2799]:   File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
Sep 23 12:41:00 capesbxhost02 python3[2799]:     self.run()
Sep 23 12:41:00 capesbxhost02 python3[2799]:   File "/usr/lib/python3.10/threading.py", line 953, in run
Sep 23 12:41:00 capesbxhost02 python3[2799]:     self._target(*self._args, **self._kwargs)
Sep 23 12:41:00 capesbxhost02 python3[2799]:   File "/opt/CAPEv2/modules/machinery/az.py", line 660, in _thr_wait_for_ready_machine
Sep 23 12:41:00 capesbxhost02 python3[2799]:     raise CuckooGuestCriticalTimeout(
Sep 23 12:41:00 capesbxhost02 python3[2799]: lib.cuckoo.common.exceptions.CuckooGuestCriticalTimeout: Machine CAPEAnalysisVMSS_1: the guest initialization hit the critical timeout, analysis aborted.
Sep 23 12:41:00 capesbxhost02 python3[2799]: 2022-09-23 12:41:00,639 [modules.machinery.az] DEBUG: Trying <bound method ResourceSkusOperations.list of <azure.mgmt.compute.v2021_07_01.operations._resource_skus_operations.ResourceSkusOperations object at 0x7f8894cbf490>>(('WestEurope',))
Sep 23 12:41:17 capesbxhost02 python3[2799]: 2022-09-23 12:41:17,681 [lib.cuckoo.core.scheduler] INFO: Loaded 1 machine/s
Sep 23 12:41:17 capesbxhost02 python3[2799]: 2022-09-23 12:41:17,685 [lib.cuckoo.core.scheduler] INFO: Waiting for analysis tasks
Sep 23 12:44:00 capesbxhost02 python3[2799]: 2022-09-23 12:44:00,884 [lib.cuckoo.core.scheduler] DEBUG: Task #1: Processing task
Sep 23 12:44:00 capesbxhost02 python3[2799]: 2022-09-23 12:44:00,886 [lib.cuckoo.core.scheduler] INFO: Task #1: Starting analysis of FILE '/tmp/cuckoo-tmp/upload_vlusaxso/cape_rubbish.txt'
Sep 23 12:44:00 capesbxhost02 python3[2799]: 2022-09-23 12:44:00,906 [lib.cuckoo.core.scheduler] INFO: Task #1: acquired machine CAPEAnalysisVMSS_1 (label=CAPEAnalysisVMSS_1, arch=x64, platform=windows)
Sep 23 12:44:00 capesbxhost02 python3[2799]: 2022-09-23 12:44:00,924 [lib.cuckoo.common.integrations.parse_pe] ERROR: PE type not recognised: 'DOS Header magic not found.'
Sep 23 12:44:00 capesbxhost02 python3[2799]: 2022-09-23 12:44:00,926 [modules.machinery.az] DEBUG: Trying <bound method UsageOperations.list of <azure.mgmt.compute.v2022_08_01.operations._usage_operations.UsageOperations object at 0x7f88652afaf0>>(('WestEurope',))
Sep 23 12:44:00 capesbxhost02 python3[2799]: 2022-09-23 12:44:00,926 [modules.machinery.az] DEBUG: Scaling CAPEAnalysisVMSS size from 0 -> 1
Sep 23 12:44:00 capesbxhost02 python3[2799]: 2022-09-23 12:44:00,927 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetsOperations.get of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_sets_operations.VirtualMachineScaleSetsOperations object at 0x7f88651f1c60>>(('it-cs_CAPESandbox_resources', 'CAPEAnalysisVMSS'))
Sep 23 12:44:00 capesbxhost02 python3[2799]: 2022-09-23 12:44:00,928 [lib.cuckoo.core.resultserver] DEBUG: Task #1: The associated machine IP is 172.21.3.7
Sep 23 12:44:00 capesbxhost02 python3[2799]: 2022-09-23 12:44:00,965 [lib.cuckoo.core.scheduler] INFO: Enabled route 'none'
Sep 23 12:44:00 capesbxhost02 python3[2799]: 2022-09-23 12:44:00,966 [lib.cuckoo.core.plugins] DEBUG: Started auxiliary module: Sniffer
Sep 23 12:44:00 capesbxhost02 python3[2799]: 2022-09-23 12:44:00,986 [lib.cuckoo.core.guest] INFO: Task #1: Starting analysis on guest (id=CAPEAnalysisVMSS_1, ip=172.21.3.7)
Sep 23 12:44:00 capesbxhost02 python3[2799]: 2022-09-23 12:44:00,991 [lib.cuckoo.core.guest] DEBUG: Task #1: CAPEAnalysisVMSS_1 is not ready yet
Sep 23 12:44:01 capesbxhost02 python3[2799]: 2022-09-23 12:44:01,096 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetsOperations.begin_update of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_sets_operations.VirtualMachineScaleSetsOperations object at 0x7f885cd49240>>(('it-cs_CAPESandbox_resources', 'CAPEAnalysisVMSS', <azure.mgmt.compute.v2022_08_01.models._models_py3.VirtualMachineScaleSet object at 0x7f88651f3e50>))
Sep 23 12:44:01 capesbxhost02 python3[2799]: 2022-09-23 12:44:01,997 [lib.cuckoo.core.guest] DEBUG: Task #1: CAPEAnalysisVMSS_1 is not ready yet
(..)
Sep 23 12:44:11 capesbxhost02 python3[2799]: 2022-09-23 12:44:11,061 [lib.cuckoo.core.guest] DEBUG: Task #1: CAPEAnalysisVMSS_1 is not ready yet
Sep 23 12:44:11 capesbxhost02 python3[2799]: 2022-09-23 12:44:11,963 [modules.machinery.az] DEBUG: The scaling of CAPEAnalysisVMSS took 11s
Sep 23 12:44:11 capesbxhost02 python3[2799]: 2022-09-23 12:44:11,963 [modules.machinery.az] DEBUG: Updated CAPEAnalysisVMSS capacity: 1; Initial capacity: 0
Sep 23 12:44:11 capesbxhost02 python3[2799]: 2022-09-23 12:44:11,963 [modules.machinery.az] DEBUG: Adding machines to database for CAPEAnalysisVMSS.
Sep 23 12:44:11 capesbxhost02 python3[2799]: 2022-09-23 12:44:11,966 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetVMsOperations.list of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_set_vms_operations.VirtualMachineScaleSetVMsOperations object at 0x7f88651f1000>>(('it-cs_CAPESandbox_resources', 'CAPEAnalysisVMSS'))
Sep 23 12:44:11 capesbxhost02 python3[2799]: 2022-09-23 12:44:11,967 [modules.machinery.az] DEBUG: Trying <bound method NetworkInterfacesOperations.list_virtual_machine_scale_set_network_interfaces of <azure.mgmt.network.v2022_01_01.operations._operations.NetworkInterfacesOperations object at 0x7f88652694e0>>(('it-cs_CAPESandbox_resources', 'CAPEAnalysisVMSS'))
Sep 23 12:44:12 capesbxhost02 python3[2799]: 2022-09-23 12:44:12,068 [lib.cuckoo.core.guest] DEBUG: Task #1: CAPEAnalysisVMSS_1 is not ready yet
Sep 23 12:44:12 capesbxhost02 python3[2799]: 2022-09-23 12:44:12,142 [modules.machinery.az] DEBUG: Scaling CAPEAnalysisVMSS has completed.
Sep 23 12:44:13 capesbxhost02 python3[2799]: 2022-09-23 12:44:13,074 [lib.cuckoo.core.guest] DEBUG: Task #1: CAPEAnalysisVMSS_1 is not ready yet
(..)
Sep 23 12:44:56 capesbxhost02 python3[2799]: 2022-09-23 12:44:56,344 [lib.cuckoo.core.guest] DEBUG: Task #1: CAPEAnalysisVMSS_1 is not ready yet
Sep 23 12:44:57 capesbxhost02 python3[2799]: 2022-09-23 12:44:57,134 [modules.machinery.az] DEBUG: Monitoring the machine pools...
Sep 23 12:44:57 capesbxhost02 python3[2799]: 2022-09-23 12:44:57,143 [modules.machinery.az] DEBUG: Trying <bound method UsageOperations.list of <azure.mgmt.compute.v2022_08_01.operations._usage_operations.UsageOperations object at 0x7f889630ded0>>(('WestEurope',))
Sep 23 12:44:57 capesbxhost02 python3[2799]: 2022-09-23 12:44:57,143 [modules.machinery.az] DEBUG: Scaling CAPEAnalysisVMSS size from 0 -> 1
Sep 23 12:44:57 capesbxhost02 python3[2799]: 2022-09-23 12:44:57,144 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetsOperations.get of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_sets_operations.VirtualMachineScaleSetsOperations object at 0x7f8865269f30>>(('it-cs_CAPESandbox_resources', 'CAPEAnalysisVMSS'))
Sep 23 12:44:57 capesbxhost02 python3[2799]: 2022-09-23 12:44:57,179 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetsOperations.begin_update of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_sets_operations.VirtualMachineScaleSetsOperations object at 0x7f8865269690>>(('it-cs_CAPESandbox_resources', 'CAPEAnalysisVMSS', <azure.mgmt.compute.v2022_08_01.models._models_py3.VirtualMachineScaleSet object at 0x7f8865220940>))
Sep 23 12:44:57 capesbxhost02 python3[2799]: 2022-09-23 12:44:57,350 [lib.cuckoo.core.guest] DEBUG: Task #1: CAPEAnalysisVMSS_1 is not ready yet
(..)
Sep 23 12:45:07 capesbxhost02 python3[2799]: 2022-09-23 12:45:07,413 [lib.cuckoo.core.guest] DEBUG: Task #1: CAPEAnalysisVMSS_1 is not ready yet
Sep 23 12:45:07 capesbxhost02 python3[2799]: 2022-09-23 12:45:07,863 [modules.machinery.az] DEBUG: The scaling of CAPEAnalysisVMSS took 11s
Sep 23 12:45:07 capesbxhost02 python3[2799]: 2022-09-23 12:45:07,863 [modules.machinery.az] DEBUG: Updated CAPEAnalysisVMSS capacity: 1; Initial capacity: 0
Sep 23 12:45:07 capesbxhost02 python3[2799]: 2022-09-23 12:45:07,863 [modules.machinery.az] DEBUG: Adding machines to database for CAPEAnalysisVMSS.
Sep 23 12:45:07 capesbxhost02 python3[2799]: 2022-09-23 12:45:07,866 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetVMsOperations.list of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_set_vms_operations.VirtualMachineScaleSetVMsOperations object at 0x7f88651f3970>>(('it-cs_CAPESandbox_resources', 'CAPEAnalysisVMSS'))
Sep 23 12:45:07 capesbxhost02 python3[2799]: 2022-09-23 12:45:07,867 [modules.machinery.az] DEBUG: Trying <bound method NetworkInterfacesOperations.list_virtual_machine_scale_set_network_interfaces of <azure.mgmt.network.v2022_01_01.operations._operations.NetworkInterfacesOperations object at 0x7f88651f1a80>>(('it-cs_CAPESandbox_resources', 'CAPEAnalysisVMSS'))
Sep 23 12:45:07 capesbxhost02 python3[2799]: 2022-09-23 12:45:07,984 [modules.machinery.az] DEBUG: Scaling CAPEAnalysisVMSS has completed.
Sep 23 12:45:08 capesbxhost02 python3[2799]: 2022-09-23 12:45:08,419 [lib.cuckoo.core.guest] DEBUG: Task #1: CAPEAnalysisVMSS_1 is not ready yet
(..)
Sep 23 12:48:20 capesbxhost02 python3[2799]: 2022-09-23 12:48:20,644 [lib.cuckoo.core.guest] DEBUG: Task #1: CAPEAnalysisVMSS_1 is not ready yet
Sep 23 12:48:21 capesbxhost02 python3[2799]: 2022-09-23 12:48:21,645 [lib.cuckoo.core.scheduler] ERROR: Machine CAPEAnalysisVMSS_1: the guest initialization hit the critical timeout, analysis aborted
Sep 23 12:48:21 capesbxhost02 python3[2799]: Traceback (most recent call last):
Sep 23 12:48:21 capesbxhost02 python3[2799]:   File "/opt/CAPEv2/lib/cuckoo/core/scheduler.py", line 364, in launch_analysis
Sep 23 12:48:21 capesbxhost02 python3[2799]:     guest.start_analysis(options)
Sep 23 12:48:21 capesbxhost02 python3[2799]:   File "/opt/CAPEv2/lib/cuckoo/core/guest.py", line 268, in start_analysis
Sep 23 12:48:21 capesbxhost02 python3[2799]:     self.wait_available()
Sep 23 12:48:21 capesbxhost02 python3[2799]:   File "/opt/CAPEv2/lib/cuckoo/core/guest.py", line 158, in wait_available
Sep 23 12:48:21 capesbxhost02 python3[2799]:     raise CuckooGuestCriticalTimeout(
Sep 23 12:48:21 capesbxhost02 python3[2799]: lib.cuckoo.common.exceptions.CuckooGuestCriticalTimeout: Machine CAPEAnalysisVMSS_1: the guest initialization hit the critical timeout, analysis aborted
Sep 23 12:48:21 capesbxhost02 python3[2799]: 2022-09-23 12:48:21,658 [lib.cuckoo.core.plugins] DEBUG: Stopped auxiliary module: Sniffer
Sep 23 12:48:21 capesbxhost02 python3[2799]: 2022-09-23 12:48:21,658 [modules.machinery.az] DEBUG: Stopping machine 'CAPEAnalysisVMSS_1'
Sep 23 12:48:22 capesbxhost02 python3[2799]: 2022-09-23 12:48:22,797 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetsOperations.begin_reimage_all of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_sets_operations.VirtualMachineScaleSetsOperations object at 0x7f8865241690>>(('it-cs_CAPESandbox_resources', 'CAPEAnalysisVMSS', <azure.mgmt.compute.v2022_08_01.models._models_py3.VirtualMachineScaleSetVMInstanceIDs object at 0x7f8865243a60>))
Sep 23 12:48:39 capesbxhost02 python3[2799]: 2022-09-23 12:48:39,145 [modules.machinery.az] DEBUG: Reimaging instances ['1'] in CAPEAnalysisVMSS took 16s
Sep 23 12:48:41 capesbxhost02 python3[2799]: 2022-09-23 12:48:41,679 [lib.cuckoo.core.resultserver] DEBUG: Task #1: Stopped tracking machine 172.21.3.7
Sep 23 12:48:41 capesbxhost02 python3[2799]: 2022-09-23 12:48:41,711 [lib.cuckoo.core.scheduler] ERROR:
Sep 23 12:48:41 capesbxhost02 python3[2799]: Traceback (most recent call last):
Sep 23 12:48:41 capesbxhost02 python3[2799]:   File "/opt/CAPEv2/lib/cuckoo/core/scheduler.py", line 461, in run
Sep 23 12:48:41 capesbxhost02 python3[2799]:     success = self.launch_analysis()
Sep 23 12:48:41 capesbxhost02 python3[2799]:   File "/opt/CAPEv2/lib/cuckoo/core/scheduler.py", line 437, in launch_analysis
Sep 23 12:48:41 capesbxhost02 python3[2799]:     raise CuckooDeadMachine()
Sep 23 12:48:41 capesbxhost02 python3[2799]: lib.cuckoo.core.scheduler.CuckooDeadMachine
Sep 23 12:48:41 capesbxhost02 python3[2799]: 2022-09-23 12:48:41,711 [lib.cuckoo.core.scheduler] INFO: Task #1: Starting analysis of FILE '/tmp/cuckoo-tmp/upload_vlusaxso/cape_rubbish.txt'
Sep 23 12:48:41 capesbxhost02 python3[2799]: 2022-09-23 12:48:41,713 [lib.cuckoo.core.scheduler] INFO: Task #1: File already exists at '/opt/CAPEv2/storage/binaries/1/07b45b34ca3b159ea764cb3382d118943f12f9a35f14ae0cae00abdfacb9243c'
Sep 23 12:48:41 capesbxhost02 python3[2799]: 2022-09-23 12:48:41,713 [lib.cuckoo.core.scheduler] INFO: Task #1: File already exists at '/opt/CAPEv2/storage/binaries/07b45b34ca3b159ea764cb3382d118943f12f9a35f14ae0cae00abdfacb9243c'
Sep 23 12:48:41 capesbxhost02 python3[2799]: 2022-09-23 12:48:41,715 [lib.cuckoo.core.scheduler] DEBUG: Task #1: no machine available yet for machine 'None', platform 'windows' or tags '[<Tag('2','x86')>]'.
Sep 23 12:48:42 capesbxhost02 python3[2799]: 2022-09-23 12:48:42,720 [lib.cuckoo.core.scheduler] DEBUG: Task #1: no machine available yet for machine 'None', platform 'windows' or tags '[<Tag('2','x86')>]'.
Sep 23 12:48:42 capesbxhost02 python3[2799]: 2022-09-23 12:48:42,837 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetsOperations.begin_delete_instances of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_sets_operations.VirtualMachineScaleSetsOperations object at 0x7f886524ba90>>(('it-cs_CAPESandbox_resources', 'CAPEAnalysisVMSS', <azure.mgmt.compute.v2022_08_01.models._models_py3.VirtualMachineScaleSetVMInstanceIDs object at 0x7f8865241d20>))
Sep 23 12:48:43 capesbxhost02 python3[2799]: 2022-09-23 12:48:43,724 [lib.cuckoo.core.scheduler] DEBUG: Task #1: no machine available yet for machine 'None', platform 'windows' or tags '[<Tag('2','x86')>]'.
(..)
Sep 23 12:49:16 capesbxhost02 python3[2799]: 2022-09-23 12:49:16,869 [lib.cuckoo.core.scheduler] DEBUG: Task #1: no machine available yet for machine 'None', platform 'windows' or tags '[<Tag('2','x86')>]'.
Sep 23 12:49:17 capesbxhost02 python3[2799]: 2022-09-23 12:49:17,053 [modules.machinery.az] DEBUG: Deleting instances ['1'] in CAPEAnalysisVMSS took 34s
Sep 23 12:49:17 capesbxhost02 python3[2799]: 2022-09-23 12:49:17,873 [lib.cuckoo.core.scheduler] DEBUG: Task #1: no machine available yet for machine 'None', platform 'windows' or tags '[<Tag('2','x86')>]'.
(..)
Sep 23 12:49:57 capesbxhost02 python3[2799]: 2022-09-23 12:49:57,041 [lib.cuckoo.core.scheduler] DEBUG: Task #1: no machine available yet for machine 'None', platform 'windows' or tags '[<Tag('2','x86')>]'.
Sep 23 12:49:57 capesbxhost02 python3[2799]: 2022-09-23 12:49:57,136 [modules.machinery.az] DEBUG: Monitoring the machine pools...
Sep 23 12:49:57 capesbxhost02 python3[2799]: 2022-09-23 12:49:57,144 [modules.machinery.az] DEBUG: Trying <bound method UsageOperations.list of <azure.mgmt.compute.v2022_08_01.operations._usage_operations.UsageOperations object at 0x7f886524b310>>(('WestEurope',))
Sep 23 12:49:57 capesbxhost02 python3[2799]: 2022-09-23 12:49:57,145 [modules.machinery.az] DEBUG: Scaling CAPEAnalysisVMSS size from 0 -> 1
Sep 23 12:49:57 capesbxhost02 python3[2799]: 2022-09-23 12:49:57,145 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetsOperations.get of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_sets_operations.VirtualMachineScaleSetsOperations object at 0x7f885cce59c0>>(('it-cs_CAPESandbox_resources', 'CAPEAnalysisVMSS'))
Sep 23 12:49:57 capesbxhost02 python3[2799]: 2022-09-23 12:49:57,176 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetsOperations.begin_update of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_sets_operations.VirtualMachineScaleSetsOperations object at 0x7f885cce7d60>>(('it-cs_CAPESandbox_resources', 'CAPEAnalysisVMSS', <azure.mgmt.compute.v2022_08_01.models._models_py3.VirtualMachineScaleSet object at 0x7f885ccfcac0>))
Sep 23 12:49:58 capesbxhost02 python3[2799]: 2022-09-23 12:49:58,045 [lib.cuckoo.core.scheduler] DEBUG: Task #1: no machine available yet for machine 'None', platform 'windows' or tags '[<Tag('2','x86')>]'.
(..)
Sep 23 12:50:58 capesbxhost02 python3[2799]: 2022-09-23 12:50:58,313 [lib.cuckoo.core.scheduler] DEBUG: Task #1: no machine available yet for machine 'None', platform 'windows' or tags '[<Tag('2','x86')>]'.
Sep 23 12:50:59 capesbxhost02 python3[2799]: 2022-09-23 12:50:59,193 [modules.machinery.az] DEBUG: The scaling of CAPEAnalysisVMSS took 62s
Sep 23 12:50:59 capesbxhost02 python3[2799]: 2022-09-23 12:50:59,193 [modules.machinery.az] DEBUG: Updated CAPEAnalysisVMSS capacity: 1; Initial capacity: 0
Sep 23 12:50:59 capesbxhost02 python3[2799]: 2022-09-23 12:50:59,194 [modules.machinery.az] DEBUG: Adding machines to database for CAPEAnalysisVMSS.
Sep 23 12:50:59 capesbxhost02 python3[2799]: 2022-09-23 12:50:59,196 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetVMsOperations.list of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_set_vms_operations.VirtualMachineScaleSetVMsOperations object at 0x7f885ccfd780>>(('it-cs_CAPESandbox_resources', 'CAPEAnalysisVMSS'))
Sep 23 12:50:59 capesbxhost02 python3[2799]: 2022-09-23 12:50:59,197 [modules.machinery.az] DEBUG: Trying <bound method NetworkInterfacesOperations.list_virtual_machine_scale_set_network_interfaces of <azure.mgmt.network.v2022_01_01.operations._operations.NetworkInterfacesOperations object at 0x7f885ccfc160>>(('it-cs_CAPESandbox_resources', 'CAPEAnalysisVMSS'))
Sep 23 12:50:59 capesbxhost02 python3[2799]: 2022-09-23 12:50:59,317 [lib.cuckoo.core.scheduler] DEBUG: Task #1: no machine available yet for machine 'None', platform 'windows' or tags '[<Tag('2','x86')>]'.
Sep 23 12:50:59 capesbxhost02 python3[2799]: 2022-09-23 12:50:59,428 [modules.machinery.az] DEBUG: Scaling CAPEAnalysisVMSS has completed.
Sep 23 12:51:00 capesbxhost02 python3[2799]: 2022-09-23 12:51:00,333 [lib.cuckoo.core.scheduler] INFO: Task #1: acquired machine CAPEAnalysisVMSS_2 (label=CAPEAnalysisVMSS_2, arch=x64, platform=windows)
Sep 23 12:51:00 capesbxhost02 python3[2799]: 2022-09-23 12:51:00,347 [lib.cuckoo.common.integrations.parse_pe] ERROR: PE type not recognised: 'DOS Header magic not found.'
Sep 23 12:51:00 capesbxhost02 python3[2799]: 2022-09-23 12:51:00,349 [modules.machinery.az] DEBUG: Trying <bound method UsageOperations.list of <azure.mgmt.compute.v2022_08_01.operations._usage_operations.UsageOperations object at 0x7f885ccff3d0>>(('WestEurope',))
Sep 23 12:51:00 capesbxhost02 python3[2799]: 2022-09-23 12:51:00,349 [modules.machinery.az] DEBUG: Scaling CAPEAnalysisVMSS size from 0 -> 1
Sep 23 12:51:00 capesbxhost02 python3[2799]: 2022-09-23 12:51:00,350 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetsOperations.get of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_sets_operations.VirtualMachineScaleSetsOperations object at 0x7f885ccff280>>(('it-cs_CAPESandbox_resources', 'CAPEAnalysisVMSS'))
Sep 23 12:51:00 capesbxhost02 python3[2799]: 2022-09-23 12:51:00,351 [lib.cuckoo.core.resultserver] DEBUG: Task #1: The associated machine IP is 172.21.3.7
Sep 23 12:51:00 capesbxhost02 python3[2799]: 2022-09-23 12:51:00,378 [lib.cuckoo.core.scheduler] INFO: Enabled route 'none'
Sep 23 12:51:00 capesbxhost02 python3[2799]: 2022-09-23 12:51:00,378 [lib.cuckoo.core.plugins] DEBUG: Started auxiliary module: Sniffer
Sep 23 12:51:00 capesbxhost02 python3[2799]: 2022-09-23 12:51:00,389 [lib.cuckoo.core.guest] INFO: Task #1: Starting analysis on guest (id=CAPEAnalysisVMSS_2, ip=172.21.3.7)
Sep 23 12:51:00 capesbxhost02 python3[2799]: 2022-09-23 12:51:00,472 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetsOperations.begin_update of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_sets_operations.VirtualMachineScaleSetsOperations object at 0x7f885ccfff70>>(('it-cs_CAPESandbox_resources', 'CAPEAnalysisVMSS', <azure.mgmt.compute.v2022_08_01.models._models_py3.VirtualMachineScaleSet object at 0x7f885cd1b460>))
Sep 23 12:51:01 capesbxhost02 python3[2799]: 2022-09-23 12:51:01,392 [lib.cuckoo.core.guest] DEBUG: Task #1: CAPEAnalysisVMSS_2 is not ready yet
(..)
Sep 23 12:51:07 capesbxhost02 python3[2799]: 2022-09-23 12:51:07,434 [lib.cuckoo.core.guest] DEBUG: Task #1: CAPEAnalysisVMSS_2 is not ready yet

@kevoreilly
Copy link
Owner

Tcpdump requires root privileges, but since cape does not run as root you have to set specific Linux capabilities to the binary:

$ sudo setcap cap_net_raw,cap_net_admin=eip /usr/sbin/tcpdump

You can verify the results of last command with:

$ getcap /usr/sbin/tcpdump
/usr/sbin/tcpdump = cap_net_admin,cap_net_raw+eip

@cccs-kevin
Copy link
Collaborator

Thanks for the trick. The golden trick is curl <ip of guest>:8000 The problem are located on agent Task Scheduler. We are modify "Set the trigger as When I logon and click Next." to "When the compute starts" And Set from "Run only when user is logged on" to "Run whether user is logged on or not"

Feel free to update the docs with a PR for this page so that other users don't make the same mistake https://capev2.readthedocs.io/en/latest/installation/guest/agent.html

@rleal124
Copy link
Author

Tcpdump requires root privileges, but since cape does not run as root you have to set specific Linux capabilities to the binary:

$ sudo setcap cap_net_raw,cap_net_admin=eip /usr/sbin/tcpdump

You can verify the results of last command with:

$ getcap /usr/sbin/tcpdump
/usr/sbin/tcpdump = cap_net_admin,cap_net_raw+eip

On ubuntu 22.04 the location of tcpdump is under /usr/bin/tcpdump
In below the output getcap /usr/bin/tcpdump, I think the result is similar.

cape@capesbxhost02:/opt/CAPEv2$ getcap /usr/bin/tcpdump
/usr/bin/tcpdump cap_net_admin,cap_net_raw=eip

@doomedraven
Copy link
Collaborator

yes, that is changed in cape2.sh but not in aux.conf

@doomedraven
Copy link
Collaborator

about why tcpdump doesn't work no really idea, but what i can suggest is to add print statement and see args, then run them by hand in terminal to view what kind of error it gives you

@rleal124
Copy link
Author

about why tcpdump doesn't work no really idea, but what i can suggest is to add print statement and see args, then run them by hand in terminal to view what kind of error it gives you

Can you please tell me how I can do?

@rleal124
Copy link
Author

yes, that is changed in cape2.sh but not in aux.conf

Yes, I change also on aux.conf file.

# Specify the path to your local installation of tcpdump. Make sure this
# path is correct.
#tcpdump = /usr/sbin/tcpdump
tcpdump = /usr/bin/tcpdump

@doomedraven
Copy link
Collaborator

https://github.com/kevoreilly/CAPEv2/blob/master/modules/auxiliary/sniffer.py#L178
just before this line add print(pargs) you will get list of args, so you might need later to run something like this once you copy that list: "".join(WHOLE LIST OF ARGS HERE AS LIST). EExamplee: "".join(["tcpdump", "x", "y", ......])

@rleal124
Copy link
Author

https://github.com/kevoreilly/CAPEv2/blob/master/modules/auxiliary/sniffer.py#L178 just before this line add print(pargs) you will get list of args, so you might need later to run something like this once you copy that list: "".join(WHOLE LIST OF ARGS HERE AS LIST). EExamplee: "".join(["tcpdump", "x", "y", ......])

I had change the code to:

try:
    print(pargs)
    self.proc = subprocess.Popen(pargs, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False)
except (OSError, ValueError):
    log.exception("Failed to start sniffer (interface=%s, host=%s, dump path=%s)", interface, host, file_path)
    return

On cape log I can observe the sniffer auxiliary started, but I expect see the following message:
"Started sniffer with PID %d (interface=%s, host=%s, dump path=%s)"
But I don't observe.

[lib.cuckoo.core.scheduler] INFO: Enabled route 'none'
Sep 26 18:25:30 capesbxhost02 python3[1002]: 2022-09-26 18:25:30,771 [lib.cuckoo.core.plugins] DEBUG: Started auxiliary module: Sniffer
Sep 26 18:25:30 capesbxhost02 python3[1002]: 2022-09-26 18:25:30,794 [lib.cuckoo.core.guest] INFO: Task #25: Starting analysis on guest (id=CAPESBXAnalysisVMSS_1, ip=172.21.3.7)
Sep 26 18:25:30 capesbxhost02 python3[1002]: 2022-09-26 18:25:30,810 [lib.cuckoo.core.guest] INFO: Task #25: Guest is running CAPE Agent 0.11 (id=CAPESBXAnalysisVMSS_1, ip=172.21.3.7)
Sep 26 18:25:30 capesbxhost02 python3[1002]: 2022-09-26 18:25:30,951 [lib.cuckoo.core.guest] DEBUG: Task #25: Uploading analyzer to guest (id=CAPESBXAnalysisVMSS_1, ip=172.21.3.7, size=26692942)

@doomedraven
Copy link
Collaborator

doomedraven commented Sep 26, 2022 via email

@rleal124
Copy link
Author

rleal124 commented Sep 26, 2022

That wont work, do it in my way El lun, 26 sept 2022 20:36, rleal124 @.> escribió:

https://github.com/kevoreilly/CAPEv2/blob/master/modules/auxiliary/sniffer.py#L178 just before this line add print(pargs) you will get list of args, so you might need later to run something like this once you copy that list: "".join(WHOLE LIST OF ARGS HERE AS LIST). EExamplee: "".join(["tcpdump", "x", "y", ......]) I had change the code to: try: print(pargs) self.proc = subprocess.Popen(pargs, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False) except (OSError, ValueError): log.exception("Failed to start sniffer (interface=%s, host=%s, dump path=%s)", interface, host, file_path) return On cape log I can observe the sniffer auxiliary started, but I expect see the following message: "Started sniffer with PID %d (interface=%s, host=%s, dump path=%s)" But I don't observe. [lib.cuckoo.core.scheduler] INFO: Enabled route 'none' Sep 26 18:25:30 capesbxhost02 python3[1002]: 2022-09-26 18:25:30,771 [lib.cuckoo.core.plugins] DEBUG: Started auxiliary module: Sniffer Sep 26 18:25:30 capesbxhost02 python3[1002]: 2022-09-26 18:25:30,794 [lib.cuckoo.core.guest] INFO: Task #25: Starting analysis on guest (id=CAPESBXAnalysisVMSS_1, ip=172.21.3.7) Sep 26 18:25:30 capesbxhost02 python3[1002]: 2022-09-26 18:25:30,810 [lib.cuckoo.core.guest] INFO: Task #25: Guest is running CAPE Agent 0.11 (id=CAPESBXAnalysisVMSS_1, ip=172.21.3.7) Sep 26 18:25:30 capesbxhost02 python3[1002]: 2022-09-26 18:25:30,951 [lib.cuckoo.core.guest] DEBUG: Task #25: Uploading analyzer to guest (id=CAPESBXAnalysisVMSS_1, ip=172.21.3.7, size=26692942) — Reply to this email directly, view it on GitHub <#1156 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOFH33JNBIGVAYQYOGHDXTWAHUKLANCNFSM6AAAAAAQSEYVWM . You are receiving this because you were mentioned.Message ID: @.
>

I am sorry, this correct?

print(''.join[tcpdump, "-U", "-q", "-s", "0", "-i", interface, "-n"])
self.proc = subprocess.Popen(pargs, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False)

@doomedraven
Copy link
Collaborator

doomedraven commented Sep 26, 2022 via email

@rleal124
Copy link
Author

rleal124 commented Sep 26, 2022

Show me the output of pargs El lun, 26 sept 2022 21:18, rleal124 @.> escribió:

That wont work, do it in my way El lun, 26 sept 2022 20:36, rleal124 @. > escribió: … <#m_1631479041120564023_> https://github.com/kevoreilly/CAPEv2/blob/master/modules/auxiliary/sniffer.py#L178 https://github.com/kevoreilly/CAPEv2/blob/master/modules/auxiliary/sniffer.py#L178 just before this line add print(pargs) you will get list of args, so you might need later to run something like this once you copy that list: "".join(WHOLE LIST OF ARGS HERE AS LIST). EExamplee: "".join(["tcpdump", "x", "y", ......]) I had change the code to: try: print(pargs) self.proc = subprocess.Popen(pargs, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False) except (OSError, ValueError): log.exception("Failed to start sniffer (interface=%s, host=%s, dump path=%s)", interface, host, file_path) return On cape log I can observe the sniffer auxiliary started, but I expect see the following message: "Started sniffer with PID %d (interface=%s, host=%s, dump path=%s)" But I don't observe. [lib.cuckoo.core.scheduler] INFO: Enabled route 'none' Sep 26 18:25:30 capesbxhost02 python3[1002]: 2022-09-26 18:25:30,771 [lib.cuckoo.core.plugins] DEBUG: Started auxiliary module: Sniffer Sep 26 18:25:30 capesbxhost02 python3[1002]: 2022-09-26 18:25:30,794 [lib.cuckoo.core.guest] INFO: Task #25 <#25>: Starting analysis on guest (id=CAPESBXAnalysisVMSS_1, ip=172.21.3.7) Sep 26 18:25:30 capesbxhost02 python3[1002]: 2022-09-26 18:25:30,810 [lib.cuckoo.core.guest] INFO: Task #25 <#25>: Guest is running CAPE Agent 0.11 (id=CAPESBXAnalysisVMSS_1, ip=172.21.3.7) Sep 26 18:25:30 capesbxhost02 python3[1002]: 2022-09-26 18:25:30,951 [lib.cuckoo.core.guest] DEBUG: Task #25 <#25>: Uploading analyzer to guest (id=CAPESBXAnalysisVMSS_1, ip=172.21.3.7, size=26692942) — Reply to this email directly, view it on GitHub <#1156 (comment) <#1156 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOFH33JNBIGVAYQYOGHDXTWAHUKLANCNFSM6AAAAAAQSEYVWM https://github.com/notifications/unsubscribe-auth/AAOFH33JNBIGVAYQYOGHDXTWAHUKLANCNFSM6AAAAAAQSEYVWM . You are receiving this because you were mentioned.Message ID: @.> I am sorry, this correct? print(''.join[tcpdump, "-U", "-q", "-s", "0", "-i", interface, "-n"]]) self.proc = subprocess.Popen(pargs, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False) — Reply to this email directly, view it on GitHub <#1156 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOFH326HHGEATUOYFAIDR3WAHZGTANCNFSM6AAAAAAQSEYVWM . You are receiving this because you were mentioned.Message ID: @.
>

So I change the code to, is correct?

 else:
            try:
                print(''.join[tcpdump, "-U", "-q", "-s", "0", "-i", interface, "-n"])
                self.proc = subprocess.Popen(pargs, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False)
            except (OSError, ValueError):
                log.exception("Failed to start sniffer (interface=%s, host=%s, dump path=%s)", interface, host, file_path)
                return

            log.info("Started sniffer with PID %d (interface=%s, host=%s, dump path=%s)", self.proc.pid, interface, host, file_path)

I should see the output on journalctl -u cape-processor.service -f correct?

@doomedraven
Copy link
Collaborator

no that is totally wrong, just put print(pargs), another print was second step outside of the cape. Is in cape not cape-procesor service, so as you doing debug stop cape service with systemctl stop cape and start it by hand with poetry run python3 cuckoo.py and once you solved the issue go back to servicee

@rleal124
Copy link
Author

rleal124 commented Sep 26, 2022

I am sorry.
First on sniffer.py add

try:
                print(pargs)
                self.proc = subprocess.Popen(pargs, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False)

Second:
systemctl stop cape

And last:
poetry run python3 cuckoo.py -d

Correct?

@doomedraven
Copy link
Collaborator

doomedraven commented Sep 26, 2022 via email

@rleal124
Copy link
Author

poetry run python3 cuckoo.py -d Output, I don't observe the pargs.

2022-09-26 20:19:19,991 [lib.cuckoo.core.scheduler] INFO: Task #34: Starting analysis of URL 'sapo.pt'
2022-09-26 20:19:20,010 [lib.cuckoo.core.scheduler] INFO: Task #34: acquired machine CAPESBXAnalysisVMSS_1 (label=CAPESBXAnalysisVMSS_1, arch=x64, platform=windows)
2022-09-26 20:19:20,023 [lib.cuckoo.core.resultserver] DEBUG: Task #34: The associated machine IP is 172.21.3.7
2022-09-26 20:19:20,051 [modules.machinery.az] DEBUG: Trying <bound method UsageOperations.list of <azure.mgmt.compute.v2022_08_01.operations._usage_operations.UsageOperations object at 0x7fc2453f0e80>>(('WestEurope',))
2022-09-26 20:19:20,060 [lib.cuckoo.core.scheduler] INFO: Enabled route 'none'
2022-09-26 20:19:20,061 [lib.cuckoo.core.plugins] DEBUG: Started auxiliary module: Sniffer
2022-09-26 20:19:20,075 [lib.cuckoo.core.guest] INFO: Task #34: Starting analysis on guest (id=CAPESBXAnalysisVMSS_1, ip=172.21.3.7)
2022-09-26 20:19:20,088 [lib.cuckoo.core.guest] INFO: Task #34: Guest is running CAPE Agent 0.11 (id=CAPESBXAnalysisVMSS_1, ip=172.21.3.7)
2022-09-26 20:19:20,119 [modules.machinery.az] DEBUG: Scaling CAPESBXAnalysisVMSS size from 0 -> 1
2022-09-26 20:19:20,120 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetsOperations.get of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_sets_operations.VirtualMachineScaleSetsOperations object at 0x7fc2453faec0>>(('it-cs_CAPESandbox_resources', 'CAPESBXAnalysisVMSS'))
2022-09-26 20:19:20,176 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetsOperations.begin_update of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_sets_operations.VirtualMachineScaleSetsOperations object at 0x7fc25de52650>>(('it-cs_CAPESandbox_resources', 'CAPESBXAnalysisVMSS', <azure.mgmt.compute.v2022_08_01.models._models_py3.VirtualMachineScaleSet object at 0x7fc25de53010>))
2022-09-26 20:19:20,194 [lib.cuckoo.core.guest] DEBUG: Task #34: Uploading analyzer to guest (id=CAPESBXAnalysisVMSS_1, ip=172.21.3.7, size=26692942)
2022-09-26 20:19:20,743 [lib.cuckoo.core.guest] INFO: Task #34: Uploading support files to guest (id=CAPESBXAnalysisVMSS_1, ip=172.21.3.7)
2022-09-26 20:19:20,743 [lib.cuckoo.core.guest] INFO: Task #34: Uploading script files to guest (id=CAPESBXAnalysisVMSS_1, ip=172.21.3.7)
2022-09-26 20:19:25,832 [lib.cuckoo.core.guest] DEBUG: Task #34: Analysis is still running (id=CAPESBXAnalysisVMSS_1, ip=172.21.3.7)
2022-09-26 20:19:31,696 [modules.machinery.az] DEBUG: The scaling of CAPESBXAnalysisVMSS took 12s
2022-09-26 20:19:31,697 [modules.machinery.az] DEBUG: Updated CAPESBXAnalysisVMSS capacity: 1; Initial capacity: 0
2022-09-26 20:19:31,697 [modules.machinery.az] DEBUG: Adding machines to database for CAPESBXAnalysisVMSS.
2022-09-26 20:19:31,700 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetVMsOperations.list of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_set_vms_operations.VirtualMachineScaleSetVMsOperations object at 0x7fc25de61420>>(('it-cs_CAPESandbox_resources', 'CAPESBXAnalysisVMSS'))
2022-09-26 20:19:31,700 [modules.machinery.az] DEBUG: Trying <bound method NetworkInterfacesOperations.list_virtual_machine_scale_set_network_interfaces of <azure.mgmt.network.v2022_01_01.operations._operations.NetworkInterfacesOperations object at 0x7fc25de60a90>>(('it-cs_CAPESandbox_resources', 'CAPESBXAnalysisVMSS'))
2022-09-26 20:19:31,932 [modules.machinery.az] DEBUG: Scaling CAPESBXAnalysisVMSS has completed.
/usr/lib/python3.10/os.py:1029: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
  return io.open(fd, mode, buffering, encoding, *args, **kwargs)
2022-09-26 20:19:32,639 [lib.cuckoo.core.resultserver] DEBUG: Task #34: live log analysis.log initialized
2022-09-26 20:19:35,942 [lib.cuckoo.core.guest] DEBUG: Task #34: Analysis is still running (id=CAPESBXAnalysisVMSS_1, ip=172.21.3.7)
2022-09-26 20:19:43,343 [lib.cuckoo.core.resultserver] DEBUG: Task #34: Trying to upload file aux/usage.log
2022-09-26 20:19:43,343 [lib.cuckoo.core.resultserver] DEBUG: Task #34: Uploaded file aux/usage.log of length: 0
2022-09-26 20:19:46,049 [lib.cuckoo.core.guest] DEBUG: Task #34: Analysis is still running (id=CAPESBXAnalysisVMSS_1, ip=172.21.3.7)
2022-09-26 20:20:02,835 [lib.cuckoo.core.resultserver] DEBUG: Task #34 is sending a BSON stream. For pid 7164
2022-09-26 20:20:02,954 [lib.cuckoo.core.resultserver] DEBUG: Task #34 is sending a BSON stream. For pid 3872
2022-09-26 20:20:06,262 [lib.cuckoo.core.guest] DEBUG: Task #34: Analysis is still running (id=CAPESBXAnalysisVMSS_1, ip=172.21.3.7)
2022-09-26 20:22:16,263 [modules.machinery.az] DEBUG: Monitoring the machine pools...
2022-09-26 20:22:16,272 [modules.machinery.az] DEBUG: Trying <bound method UsageOperations.list of <azure.mgmt.compute.v2022_08_01.operations._usage_operations.UsageOperations object at 0x7fc27854dcc0>>(('WestEurope',))
2022-09-26 20:22:16,511 [modules.machinery.az] DEBUG: Scaling CAPESBXAnalysisVMSS size from 0 -> 1
2022-09-26 20:22:16,512 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetsOperations.get of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_sets_operations.VirtualMachineScaleSetsOperations object at 0x7fc27854dc60>>(('it-cs_CAPESandbox_resources', 'CAPESBXAnalysisVMSS'))
2022-09-26 20:22:16,556 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetsOperations.begin_update of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_sets_operations.VirtualMachineScaleSetsOperations object at 0x7fc25dead6f0>>(('it-cs_CAPESandbox_resources', 'CAPESBXAnalysisVMSS', <azure.mgmt.compute.v2022_08_01.models._models_py3.VirtualMachineScaleSet object at 0x7fc25deadd20>))
2022-09-26 20:22:17,637 [lib.cuckoo.core.guest] DEBUG: Task #34: Analysis is still running (id=CAPESBXAnalysisVMSS_1, ip=172.21.3.7)
2022-09-26 20:22:27,711 [modules.machinery.az] DEBUG: The scaling of CAPESBXAnalysisVMSS took 11s
2022-09-26 20:22:27,712 [modules.machinery.az] DEBUG: Updated CAPESBXAnalysisVMSS capacity: 1; Initial capacity: 0
2022-09-26 20:22:27,713 [modules.machinery.az] DEBUG: Adding machines to database for CAPESBXAnalysisVMSS.
2022-09-26 20:22:27,716 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetVMsOperations.list of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_set_vms_operations.VirtualMachineScaleSetVMsOperations object at 0x7fc25deafa90>>(('it-cs_CAPESandbox_resources', 'CAPESBXAnalysisVMSS'))
2022-09-26 20:22:27,716 [modules.machinery.az] DEBUG: Trying <bound method NetworkInterfacesOperations.list_virtual_machine_scale_set_network_interfaces of <azure.mgmt.network.v2022_01_01.operations._operations.NetworkInterfacesOperations object at 0x7fc25deaf9d0>>(('it-cs_CAPESandbox_resources', 'CAPESBXAnalysisVMSS'))
2022-09-26 20:22:27,745 [lib.cuckoo.core.guest] DEBUG: Task #34: Analysis is still running (id=CAPESBXAnalysisVMSS_1, ip=172.21.3.7)
2022-09-26 20:22:28,003 [modules.machinery.az] DEBUG: Scaling CAPESBXAnalysisVMSS has completed.
2022-09-26 20:22:32,801 [lib.cuckoo.core.guest] DEBUG: Task #34: Analysis is still running (id=CAPESBXAnalysisVMSS_1, ip=172.21.3.7)
2022-09-26 20:23:22,931 [lib.cuckoo.core.resultserver] DEBUG: Task #34: Trying to upload file procdump/6ea6f9fbda989ebf3aaae3041bdddaf84c8a0ec0cd867706b318ebb562e88be3
2022-09-26 20:23:22,942 [lib.cuckoo.core.resultserver] DEBUG: Task #34: Uploaded file procdump/6ea6f9fbda989ebf3aaae3041bdddaf84c8a0ec0cd867706b318ebb562e88be3 of length: 827904
2022-09-26 20:23:23,360 [lib.cuckoo.core.guest] DEBUG: Task #34: Analysis is still running (id=CAPESBXAnalysisVMSS_1, ip=172.21.3.7)
2022-09-26 20:23:24,417 [lib.cuckoo.core.resultserver] DEBUG: Task #34: Trying to upload file curtain/1664223804.3878918.curtain.log
2022-09-26 20:23:24,668 [lib.cuckoo.core.resultserver] DEBUG: Task #34: Uploaded file curtain/1664223804.3878918.curtain.log of length: 12083764
2022-09-26 20:23:25,380 [lib.cuckoo.core.guest] INFO: Task #34: Analysis completed successfully (id=CAPESBXAnalysisVMSS_1, ip=172.21.3.7)
2022-09-26 20:23:25,405 [lib.cuckoo.core.plugins] DEBUG: Stopped auxiliary module: Sniffer
2022-09-26 20:23:25,405 [modules.machinery.az] DEBUG: Stopping machine 'CAPESBXAnalysisVMSS_1'
2022-09-26 20:23:25,729 [lib.cuckoo.core.resultserver] DEBUG: Task #34 had connection reset by peer for <Context for b'LOG'>
2022-09-26 20:23:25,730 [lib.cuckoo.core.resultserver] DEBUG: Task #34 had connection reset by peer for <Context for b'BSON'>
2022-09-26 20:23:25,730 [lib.cuckoo.core.resultserver] DEBUG: Task #34 had connection reset by peer for <Context for b'BSON'>
2022-09-26 20:23:25,731 [lib.cuckoo.core.resultserver] DEBUG: Task #34 had connection reset by peer for <Context for None>
2022-09-26 20:23:28,297 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetsOperations.begin_reimage_all of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_sets_operations.VirtualMachineScaleSetsOperations object at 0x7fc25de10a60>>(('it-cs_CAPESandbox_resources', 'CAPESBXAnalysisVMSS', <azure.mgmt.compute.v2022_08_01.models._models_py3.VirtualMachineScaleSetVMInstanceIDs object at 0x7fc2453f3010>))
2022-09-26 20:23:42,516 [modules.machinery.az] DEBUG: Reimaging instances ['1'] in CAPESBXAnalysisVMSS took 14s
2022-09-26 20:23:45,434 [lib.cuckoo.core.resultserver] DEBUG: Task #34: Stopped tracking machine 172.21.3.7
2022-09-26 20:23:45,474 [lib.cuckoo.core.scheduler] DEBUG: Task #34: Released database task with status True
2022-09-26 20:23:45,474 [lib.cuckoo.core.scheduler] INFO: Task #34: analysis procedure completed
2022-09-26 20:27:16,265 [modules.machinery.az] DEBUG: Monitoring the machine pools...
2022-09-26 20:27:16,277 [modules.machinery.az] DEBUG: Trying <bound method UsageOperations.list of <azure.mgmt.compute.v2022_08_01.operations._usage_operations.UsageOperations object at 0x7fc25de63190>>(('WestEurope',))
2022-09-26 20:27:16,565 [modules.machinery.az] DEBUG: Scaling CAPESBXAnalysisVMSS size from 0 -> 1
2022-09-26 20:27:16,566 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetsOperations.get of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_sets_operations.VirtualMachineScaleSetsOperations object at 0x7fc25decbe20>>(('it-cs_CAPESandbox_resources', 'CAPESBXAnalysisVMSS'))
2022-09-26 20:27:16,608 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetsOperations.begin_update of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_sets_operations.VirtualMachineScaleSetsOperations object at 0x7fc25dedaf20>>(('it-cs_CAPESandbox_resources', 'CAPESBXAnalysisVMSS', <azure.mgmt.compute.v2022_08_01.models._models_py3.VirtualMachineScaleSet object at 0x7fc25dedb550>))
2022-09-26 20:27:27,683 [modules.machinery.az] DEBUG: The scaling of CAPESBXAnalysisVMSS took 11s
2022-09-26 20:27:27,690 [modules.machinery.az] DEBUG: Updated CAPESBXAnalysisVMSS capacity: 1; Initial capacity: 0
2022-09-26 20:27:27,690 [modules.machinery.az] DEBUG: Adding machines to database for CAPESBXAnalysisVMSS.
2022-09-26 20:27:27,694 [modules.machinery.az] DEBUG: Trying <bound method VirtualMachineScaleSetVMsOperations.list of <azure.mgmt.compute.v2022_08_01.operations._virtual_machine_scale_set_vms_operations.VirtualMachineScaleSetVMsOperations object at 0x7fc2453d0ca0>>(('it-cs_CAPESandbox_resources', 'CAPESBXAnalysisVMSS'))
2022-09-26 20:27:27,696 [modules.machinery.az] DEBUG: Trying <bound method NetworkInterfacesOperations.list_virtual_machine_scale_set_network_interfaces of <azure.mgmt.network.v2022_01_01.operations._operations.NetworkInterfacesOperations object at 0x7fc2453d0f10>>(('it-cs_CAPESandbox_resources', 'CAPESBXAnalysisVMSS'))
2022-09-26 20:27:27,903 [modules.machinery.az] DEBUG: Scaling CAPESBXAnalysisVMSS has completed.

@rleal124
Copy link
Author

I am expecte some message like: Started sniffer with PID %d (interface=%s, host=%s, dump path=%s)
I have a feeling the sniffer (tcpdump) don't starting as expected.

@doomedraven
Copy link
Collaborator

2022-09-26 20:19:20,060 [lib.cuckoo.core.scheduler] INFO: Enabled route 'none' <- this fully disables tcpdump

@rleal124
Copy link
Author

Thanks a lot.
I need edit the routing.conf correct?

@doomedraven
Copy link
Collaborator

yes as there is no reason to use tcpdump without routing, ensure to restart cuckoo.py

@rleal124
Copy link
Author

Thanks a lot for your help and patience.
I change the config to:

[routing]
route = internet
internet = eth0

Again thanks a lot.

@rleal124
Copy link
Author

About the pargs output:

['/usr/bin/tcpdump', '-U', '-q', '-s', '0', '-i', 'eth1', '-n', '-Z', 'cape', '-w', '/opt/CAPEv2/storage/analyses/43/dump.pcap', 'host', '172.21.3.7', 'and', 'not', '(', 'dst', 'host', '172.21.3.7', 'and', 'dst', 'port', '8000', ')', 'and', 'not', '(', 'src', 'host', '172.21.3.7', 'and', 'src', 'port', '8000', ')', 'and', 'not', '(', 'dst', 'host', '172.21.0.6', 'and', 'dst', 'port', '2042', ')', 'and', 'not', '(', 'src', 'host', '172.21.0.6', 'and', 'src', 'port', '2042', ')', 'and', '(', 'not arp', ')']

@doomedraven
Copy link
Collaborator

doomedraven commented Sep 27, 2022

" ".join(['/usr/bin/tcpdump', '-U', '-q', '-s', '0', '-i', 'eth1', '-n', '-Z', 'cape', '-w', '/opt/CAPEv2/storage/analyses/43/dump.pcap', 'host', '172.21.3.7', 'and', 'not', '(', 'dst', 'host', '172.21.3.7', 'and', 'dst', 'port', '8000', ')', 'and', 'not', '(', 'src', 'host', '172.21.3.7', 'and', 'src', 'port', '8000', ')', 'and', 'not', '(', 'dst', 'host', '172.21.0.6', 'and', 'dst', 'port', '2042', ')', 'and', 'not', '(', 'src', 'host', '172.21.0.6', 'and', 'src', 'port', '2042', ')', 'and', '(', 'not arp', ')'])


# just need to add " for rule, added by me
/usr/bin/tcpdump -U -q -s 0 -i eth1 -n -Z cape -w /opt/CAPEv2/storage/analyses/43/dump.pcap "host 172.21.3.7 and not ( dst host 172.21.3.7 and dst port 8000 ) and not ( src host 172.21.3.7 and src port 8000 ) and not ( dst host 172.21.0.6 and dst port 2042 ) and not ( src host 172.21.0.6 and src port 2042 ) and ( not arp )"

run the last command by hand as cape to see what error it will give you

@rleal124
Copy link
Author

I don't get any error, but also I don't get any traffic.

I run

/usr/bin/tcpdump -U -q -s 0 -i any -n -Z cape -w /opt/CAPEv2/storage/analyses/43/dump.pcap "host 172.21.3.7 and not ( dst host 172.21.3.7 and dst port 8000 ) and not ( src host
 172.21.3.7 and src port 8000 ) and not ( dst host 172.21.0.6 and dst port 2042 ) and not ( src host 172.21.0.6 and src port 2042 ) and ( not arp )"
tcpdump: listening on eth0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
^C0 packets captured
2 packets received by filter
0 packets dropped by kernel

@doomedraven
Copy link
Collaborator

you did modify my command and is not correct, you set to listen on all, you need to run exact command

@rleal124
Copy link
Author

Sorry, I change the interface to any, because I don't get any traffic on eth1 interface. In below the output.

 /usr/bin/tcpdump -U -q -s 0 -i eth1 -n -Z cape -w /opt/CAPEv2/storage/analyses/43/dump.pcap "host 172.21.3.7 and not ( dst host 172.21.3.7 and dst port 8000 ) and not ( src host 172.21.3.7 and src port 8000 ) and not ( dst host 172.21.0.6 and dst port 2042 ) and not ( src host 172.21.0.6 and src port 2042 ) and ( not arp )"
tcpdump: listening on eth1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
^C0 packets captured
0 packets received by filter
0 packets dropped by kernel

@doomedraven
Copy link
Collaborator

well then all works as expected with tcpdump, the problem here idk what it is as i don't have azure instance. so you probably will need to figurate on your own + pay attention that iface that Kevin told you is not eth1

@rleal124
Copy link
Author

Thanks a lot for your help.

Let read again the documentation.
Many thanks for your help.

@doomedraven
Copy link
Collaborator

doomedraven commented Oct 11, 2022 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants