All analyses stuck in "pending" and never run #268
Comments
|
Hello, can you post output?
python3 cuckoo.py - d
El vie., 11 sept. 2020 20:18, Pieter Bork <[email protected]>
escribió:
… This is opensource and you getting *free* support so be friendly!
Prerequisites
Please answer the following questions for yourself before submitting an
issue.
- I am running the latest version
- I checked the documentation and found no answer
- I checked to make sure that this issue has not already been filed
- I'm reporting the issue to the correct repository (for
multi-repository projects)
Expected Behavior
Files are submitted to VMs and results are processed
Current Behavior
All files submitted create tasks that are "pending" forever
Failure Information (for bugs)
I've recently upgraded from CapeV1 to CapeV2 and haven't been able to get
analyses to process properly. All files I submit end up in "pending" status
forever. I am able to manually query select * from tasks; in postgresql
and see the tasks are present, but they never move into "running" status.
All services: cape-rooter, cape-processor, cape, cape-web, postgresql,
mongod are running and produce no errors. I'm currently running cape +
cape-processor with the --debug flags, but see no processing happening in
logs when I submit a file other than the web API "POST /submit/ HTTP/1.1"
200 3309
Some results from the API:
/api/machines/list
{
"data": [
{
"id": 7,
"name": "cape001",
"label": "cape001",
"ip": "192.168.123.10",
"platform": "windows",
"interface": "virbr123",
"snapshot": "cape001-snap01",
"locked": false,
"locked_changed_on": null,
"status": "poweroff",
"status_changed_on": "2020-09-11 17:57:09",
"resultserver_ip": "192.168.123.1",
"resultserver_port": "2042",
"tags": []
}
],
"error": false
}
/api/cuckoo/status
{
"error": false,
"data": {
"version": "2.1-CAPE",
"hostname": "capev2",
"machines": {
"total": 1,
"available": 1
},
"tasks": {
"total": 4,
"pending": 4,
"running": 0,
"completed": 0,
"reported": 0
}
}
}
I'm starting to go through some code, but if you have any ideas for how I
can troubleshoot this I would be very appreciative.
Cheers,
Pieter
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#268>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAOFH32VH3F6672PJNFHLN3SFJSWVANCNFSM4RIBMVYQ>
.
|
|
Here is the output from |
|
how do you adding tasks? i suppouse you run something as root and something as another user, that is most common issue for pending tasks |
|
I'm submitting using the web UI, only the rooter is running as root. All submissions with the Web UI are stuck "pending". Trying to submit with submit.py seems to actually work to submit the analysis But then the web UI is broken trying to go to the "Recent" tab, throwing this error: Thanks for all your help and hard work on this project! |
|
Well then i suppouse ypu have some permission problem as we using cape
username and not cuckoo, chekc if all folders and tmp folder is under the
same username
El mar., 15 sept. 2020 0:42, Pieter Bork <[email protected]>
escribió:
… I'm submitting using the web UI, only the rooter is running as root.
root 64778 1 0 21:42 ? 00:00:00 /usr/bin/python3 rooter.py -g cuckoo
cuckoo 69483 1 0 21:45 ? 00:00:05 /usr/bin/python3 process.py -p7 auto -pt 900 --debug
cuckoo 69510 1 1 21:45 ? 00:00:29 /usr/bin/python3 cuckoo.py --debug
cuckoo 69511 1 0 21:45 ? 00:00:02 /usr/bin/python3 manage.py runserver 0.0.0.0:8000
postgres 69534 63011 0 21:45 ? 00:00:00 postgres: cuckoo analysis 127.0.0.1(47548) idle
postgres 69573 63011 0 21:45 ? 00:00:04 postgres: cuckoo analysis 127.0.0.1(47558) idle
postgres 69574 63011 0 21:45 ? 00:00:00 postgres: cuckoo analysis 127.0.0.1(47560) idle
cuckoo 69581 69511 1 21:45 ? 00:00:42 /usr/bin/python3 manage.py runserver 0.0.0.0:8000
postgres 69627 63011 0 21:45 ? 00:00:00 postgres: cuckoo analysis 127.0.0.1(47580) idle
Trying to submit with submit.py seems to actually work to submit the
analysis
***@***.*** opt]# sudo -u cuckoo python3 CAPEv2/utils/submit.py 46224284f2cc1537456bc385e85b1a0ef9a93b49
Success: File "/opt/46224284f2cc1537456bc385e85b1a0ef9a93b49" added as task with ID 2
2020-09-14 22:33:38,705 [lib.cuckoo.core.scheduler] DEBUG: Task #2: Processing task
2020-09-14 22:33:38,707 [lib.cuckoo.core.scheduler] INFO: Task #2: Starting analysis of FILE '/opt/46224284f2cc1537456bc385e85b1a0ef9a93b49'
2020-09-14 22:33:38,720 [lib.cuckoo.core.scheduler] INFO: Task #2: acquired machine cape001 (label=cape001, platform=windows)
2020-09-14 22:33:38,746 [root] DEBUG: Now tracking machine 192.168.123.10 for task #2
2020-09-14 22:33:38,756 [lib.cuckoo.common.abstracts] DEBUG: Starting machine cape001
2020-09-14 22:33:38,756 [lib.cuckoo.common.abstracts] DEBUG: Getting status for cape001
2020-09-14 22:33:38,789 [lib.cuckoo.common.abstracts] DEBUG: Using snapshot cape001-snap01 for virtual machine cape001
2020-09-14 22:33:40,519 [lib.cuckoo.common.abstracts] DEBUG: Getting status for cape001
2020-09-14 22:33:40,553 [lib.cuckoo.core.scheduler] INFO: Enabled route 'none'
2020-09-14 22:33:40,561 [modules.auxiliary.sniffer] INFO: Started sniffer with PID 71495 (interface=virbr123, host=192.168.123.10, dump path=/opt/CAPEv2/storage/analyses/2/dump.pcap)
2020-09-14 22:33:40,562 [lib.cuckoo.core.plugins] DEBUG: Started auxiliary module: Sniffer
2020-09-14 22:33:40,576 [lib.cuckoo.core.guest] INFO: Starting analysis #2 on guest (id=cape001, ip=192.168.123.10)
2020-09-14 22:33:41,579 [lib.cuckoo.core.guest] DEBUG: cape001: not ready yet
2020-09-14 22:33:42,582 [lib.cuckoo.core.guest] DEBUG: cape001: not ready yet
2020-09-14 22:33:43,585 [lib.cuckoo.core.guest] DEBUG: cape001: not ready yet
2020-09-14 22:33:43,594 [lib.cuckoo.core.guest] INFO: Guest is running CAPE Agent 0.11 (id=cape001, ip=192.168.123.10)
2020-09-14 22:33:43,629 [lib.cuckoo.core.guest] DEBUG: Uploading analyzer to guest (id=cape001, ip=192.168.123.10, size=17059709)
2020-09-14 22:33:43,988 [lib.cuckoo.core.guest] INFO: Uploading support files to guest (id=cape001, ip=192.168.123.10)
2020-09-14 22:33:54,933 [lib.cuckoo.core.guest] DEBUG: cape001: analysis #2 is still running
2020-09-14 22:33:59,957 [lib.cuckoo.core.guest] DEBUG: cape001: analysis #2 is still running
...
2020-09-14 22:38:10,458 [lib.cuckoo.core.guest] INFO: cape001: end of analysis reached!
2020-09-14 22:38:10,466 [lib.cuckoo.core.plugins] DEBUG: Stopped auxiliary module: Sniffer
2020-09-14 22:38:10,466 [lib.cuckoo.common.abstracts] DEBUG: Stopping machine cape001
2020-09-14 22:38:10,466 [lib.cuckoo.common.abstracts] DEBUG: Getting status for cape001
2020-09-14 22:38:10,701 [lib.cuckoo.common.abstracts] DEBUG: Getting status for cape001
2020-09-14 22:38:10,722 [root] DEBUG: Stopped tracking machine 192.168.123.10 for task #2
2020-09-14 22:38:10,792 [lib.cuckoo.core.scheduler] DEBUG: Task #2: Released database task with status True
2020-09-14 22:38:10,792 [lib.cuckoo.core.scheduler] INFO: Task #2: analysis procedure completed
But then the web UI is broken trying to go to the "Recent" tab, throwing
this error:
OperationFailure at /analysis/
Path collision at info.custom remaining portion custom, full error: {'ok': 0.0, 'errmsg': 'Path collision at info.custom remaining portion custom', 'code': 31249, 'codeName': 'Location31249'}
Thanks for all your help and hard work on this project!
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#268 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAOFH3YJNCUZBEXAUPSTALDSF2L43ANCNFSM4RIBMVYQ>
.
|
|
I've updated my installation to use the cape user instead of the cuckoo user and reviewed your install scripts to try to find anything else I may be missing, but still having the same issues. Here are the groups/process/files that are cape-related: Submissions with Web UI are still stuck in pending. If I submit with Here are the tasks from the mongo/postgres: I agree that this seems like a permissions issue, but I'm having issues figuring it out. Any suggestions of other things I can check? |
|
is first time that i see that mongo error, but that is probably part of your problem, ensure that mongo works correctly, for othe rest everything looks correct |
|
I commented out Lines 118-138 in web/analysis/view.py which fixed the pymongo error being thrown. I've found that if I use "Detect Automatically" selected in the web UI, my tasks will be "pending" forever. However, selecting my VM from the dropdown seems to work and my sample processes successfully. I'm going to do some more testing with tags and different VM names and will report back any additional findings. |
|
So from what you telling it sound line your sample is x64 but you didnt set
tags in machinery conf?
El mié., 16 sept. 2020 23:26, Pieter Bork <[email protected]>
escribió:
… I commented out Lines 118-138 in web/analysis/view.py
<https://github.com/kevoreilly/CAPEv2/blob/1f534ffcd6c4ed7ca0574f6f75a5381943259321/web/analysis/views.py#L118-L138>
which fixed the pymongo error being thrown.
I've found that if I use "Detect Automatically" selected in the web UI, my
tasks will be "pending" forever. However, selecting my VM from the dropdown
seems to work and my sample processes successfully.
I'm going to do some more testing with tags and different VM names and
will report back any additional findings.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#268 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAOFH34LJWLMOEMZEEZSBGLSGEURXANCNFSM4RIBMVYQ>
.
|
|
Yep, that was why Detect Automatically was not working. Last issue I'm having is that Cape doesn't seem to be able to start virtual machines by itself when tasks are acquired - I have to start the machine manually for processing to continue. If I do nothing: After running Seems like cape is missing permissions to start VMs, but this seems to work fine: If I add |
|
go to the old cape repo and search in issues, or maybe it was cuckoo-modified, i don't remember already, and you will see that create is required when you incorrectly created vm |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This is opensource and you getting free support so be friendly!
Prerequisites
Please answer the following questions for yourself before submitting an issue.
Expected Behavior
Files are submitted to VMs and results are processed
Current Behavior
All files submitted create tasks that are "pending" forever
Failure Information (for bugs)
I've recently upgraded from CapeV1 to CapeV2 and haven't been able to get analyses to process properly. All files I submit end up in "pending" status forever. I am able to manually query
select * from tasks;in postgresql and see the tasks are present, but they never move into "running" status.All services: cape-rooter, cape-processor, cape, cape-web, postgresql, mongod are running and produce no errors. I'm currently running cape + cape-processor with the --debug flags, but see no processing happening in logs when I submit a file other than the web API
"POST /submit/ HTTP/1.1" 200 3309Some results from the API:
/api/machines/list
/api/cuckoo/status
I'm starting to go through some code, but if you have any ideas for how I can troubleshoot this I would be very appreciative.
Cheers,
Pieter
The text was updated successfully, but these errors were encountered: