General feedback

[S]

I emailed this to support a few weeks ago, but never got a response, so I'll try posting it here:

I've spent the past week or so testing out SAP as well as several other security products, and it's definitely at the top of my list along with VoodooShield and a few others, but there are several issues I have with it that prevent me from feeling adequately protected without using it with VS and/or something else, as well as some minor things:

• The biggest problem I've found with it is that once a file/program is trusted, which IMO happens far too easily (and automatically), even if the file is infected it won't show up in a scan. If it's scanned via the context menu, SAP flags it, but it's completely overlooked on a system scan. This is not only a problem in the case of files that are unintentionally trusted (which, as I said, is easily done), but also in the case of a file that was explicitly trusted only to end up later being found untrustworthy. A perfect example of when this would happen is with the Avast/CCleaner server hack which led to a malicious file signed with a valid certificate being distributed, which would have been trusted by SAP and its users, only to be discovered and have the certificate revoked, but at that point the file would already be trusted and therefore it wouldn't be flagged, leaving users completely unaware. (UPDATE: I tested this again and it didn't behave the same, so I'm not sure if I mixed something up or if it's inconsistent)
• To add to the issue of files being trusted inadvertently, I don't like that all files presently on the computer are automatically trusted, ESPECIALLY considering the first point. I understand the thinking behind it, but that thinking is flawed. This should at least be an option. It took me quite a while before figuring out to untrust entire folders/drives via the context menu. I assumed (wrongly) there would be something in the program's interface that listed all trusted items and allowed the trust to be revoked.
• Additionally, there should be an option that if a file is manually allowed despite one of the conditions (certificate or scan results) being red, it should pop up one more warning (similar to VoodooShield) that makes sure the person is aware the file is a potential risk. Basically, this is an extra step to protect against notification fatigue and semi-automatically clicking allow, which would only present itself a fraction of the time SAP pops up, and at the most important times, which would make it a lot harder to ignore. Obviously, not everyone would want it, which is why it should be an option, but I think it would add a good amount of extra protection with very little extra hassle.
• It's impossible to cancel the upload of a file when a scan is requested via the context menu. This is a potentially major issue, as even a relatively small file can take a very long time to upload, not to mention using up potentially valuable bandwidth and data. This is connected to another issue I have with the program, which is that automatic uploads can't be permanantly disabled. I don't have a fast enough internet for this to be useful, or even feasible, so all I can do is limit it to 10MB daily to limit it as much as possible. Because of this, I've gone so far as to try and disable that scanner completely, which brings me to my next point...
• Another thing I don't like is that it's not clear between the two scanners (UAV and APEX) which does what. As best as I can tell, UAV is a cloud scanner which uploads hashes and/or entire files (not sure which or if both or what) and APEX is a local scanner which uses (I'm assuming) heuristics. The two really need better descriptions and better separation. As far as I can tell, they are separate (after all, they can be disabled separately), yet APEX is under "Scan Settings" which, based on the label, sounds like nothing more than settings for the UAV scanner. It's all just really confusing.
• And speaking of the cloud scanner (UAV?), which, again, I'm assuming uses hashes when possible but sometimes has to upload the file, it's disappointing that it uses so few engines. For example, scanning the CCleaner file I mentioned, SAP shows 6/12 detections, whereas VirusTotal shows 55/70. For another file, SAP makes it appear to be a false positive, showing only 1/12, whereas VT shows 20/66 (obviously, this could still be a FP, but VT's results make it much more likely to be scrutinized).
• Clicking the blue button next to "Last Complete Scan" on the left side should show the results of that scan, not start a new one.
• There should probably be options for putting it in observation mode via the system tray icon's menu ("Application Whitelisting" submenu). This is partly due to the fact that it would help make it clear that the "trust all" modes actually GIVE trust to everything run while they're active, as opposed to merely trusting them for that time but not actually giving them trust forever from that point on.
• It's unclear what the script settings (Application Whitelisting > Advanced Settings) do. I assume everything listed in that tab is whitelisted, since it's under whitelisting settings (though so is "Restricted Applications" so that logic isn't exactly bulletproof), but removing e.g. regedit doesn't stop me from running regedit.exe nor does it stop me from running a .reg file that edits the registry, so this section doesn't appear to do anything.
• The system tray icon should change when a threat has been found by the scanner, to alert the user to check it. (UPDATE: after more testing, I don't think there's been an instance where a threat was found and it didn't leave the notification in place, which makes it very obvious there's an issue, so this point is probably invalid, but I've left it just in case)

[hendy]

I apologize for my late reply.

1. For trust by trusted certificate, this setting can be adjusted. By default (https://support.secureaplus.com/how-to-view-the-application-whitelisting-settings/), the certificate is trusted as long as it is in the certificate list. https://support.secureaplus.com/how-do-i-manage-my-list-of-trusted-certificates/
For most secure setting, user can turn off the trust by digital certificate. In this case, it will only trust the file base on the hash.
We also distinguish the users by trusted and non-trusted user accounts. In the enterprise environment where most of the users are running under non-trusted user account, it will prevent the user from making the wrong decision, because non-trusted account user can only run in lockdown mode, and they don't have right to trust any new files.
For the case that user accidentally trust a file, probably at the time when it was detected as clean, when Universal AV detected it as a virus, it will notify the user.

2. After initial whitelisting completed, the hashes will be sent to the Universal AV server.
   When any of the files are detected as malware, it will prompt the user. When the user decide to quarantine or delete the detected files, the certificate will be also be automatically removed from the trusted certificate list.
   
3. This is a good suggestion. I also like the idea that this should be optional.

4. You can abort the file upload by closing the window (the cross button on the top right corner).
   You also can unchecked the "Upload the following type of unknown files to Universal AV server" (https://support.secureaplus.com/how-to-do-file-folder-scanning/), if you don't want to upload any files during the scan via context menu (manual scan).
   
5. You can click on the arrow (on the right of the item), to view the details. Once it is expanded, you will see the details which engine detect it.


6. Our engines in the cloud are not running in parallel at the same time. Some time the detection from other engines may come at later time.
   We will try to see on how to improve this in the future.
   
7. You got a point on this. I'm agree that it is make sense to show the last scan result rather than start a new one.

8. Basically this is a good idea. It can be either at the Application Whitelisting menu or in the same group with other mode (Normal mode, Silent Mode, Observation Mode).

9. Basically our application whitelisting covers excutable files and scripts.
   How do we identify executable files are by the PE header structure. For scripts, it doesn't have a specific header structure, so we identify it by the extension and the interpreter that execute the script. We put this in the settings, so that it can be easily extended in the future. For example, last time there was no powershell. Once Windows introduce powershell, we can extend the script coverage to powershell by using this setting.
   Restricted application is completely a different thing. Basically restricted application is used to prevent a file to be set as a trusted installer, either automatically or unintentionally manually set.
   For the detail, you may refer to: https://support.secureaplus.com/how-can-i-set-applications-as-restricted-application/
   
10. For Universal AV and real-time scanning, there will be a notification when threat is detected.

[S]

My reply for now will have to be somewhat limited, since I'm without internet for a while and, as such, can't do any scanning (this is why I don't like cloud-only scanners, but I guess it's a trade-off). Anyways, I'll have to play around with it some more once my internet is back up (probably another week) before I can really respond to the first two points.

4. Good to know it can be aborted by closing the window, but sometimes in cases like this programs will continue to do things in the background when such a window is closed, and there's nothing really to make the fact it will be canceled known to the user. I would strongly suggest actually adding a cancel button. As for disabling upload of unknown files, there are a couple issues I see with that. First, it only relates to context-menu scans, not system scans. Second, AFAICT it is only accessible once the scan is started, when it should be in the program's options as well (just to make it more apparent).

5. I'm not talking about determining which engine (UAV or APEX) detected an item, I'm talking about in options/settings it's unclear both what the differences are between the two and how the various settings apply to each. As I said, there is a settings heading titled "Universal AV" and one titled "Scan Settings," which, AFAICT, the first is really UAV scan settings and the second is really APEX scan settings, in which case they should be labeled that way. The current way of labeling them is just confusing. And it would be helpful to provide a short description under each, probably at the top of the first tab or even above the tabs. Something like "Universal AV (UAV) is the cloud-scanning component of SecureAPlus, which uploads hashes of files (and the files themselves when their hashes aren't in the cloud database) to be checked with multiple scanning engines from different vendors. It is used for system and context-menu/on-demand scans." and "APEX is a local, offline scanner that uses heuristics (or signatures, or however it works) for real-time protection only, and does not provide on-demand protection." Changing the labels (UAV, Scan Settings) and adding descriptions like these would make things immensely more clear when figuring out how to use SAP. Also, assuming APEX is in fact real-time only, which appears to be the case since I can't run scans with SAP without internet, then having a switch in its settings to enable/disable real-time protection doesn't make sense, as it seems to indicate there will still be other aspects of APEX active even with it disabled. If that is the case, this is where a description and perhaps some improved descriptions for each setting would oome in handy. If it's not the case, and disabling the RTP setting effectively disables APEX, then it should be labeled as such (currently its label is the only thing describing APEX as a real-time scanner, but adding the previously mentioned description at the top would cover that) and should probably be the very top-most setting and even cause other settings to gray out when it's disabled.

6. I'm not sure if you're saying you use more engines but not all of them are used at any given time (which wouldn't make any sense) and/or that you'll try to add more in the future to increase the breadth of the scans, in which case that would be great.

8. My understanding is that normal and silent modes are comparable to ringer vs vibrate/silent on a phone. That is, both behave the exact same way, the only difference is in how they interact with the user, i.e. notifications or not. Assuming that's the case (and the fact it remains in normal or silent when the application whitelisting mode is changed reinforces that assumption) then it wouldn't make sense to place observation mode with those, because each group should only be able to have one setting within it active at a time. For example, if it were placed with normal and silent modes, you would have one of those two selected and separately enable observation mode. The key word there is "separately." So it should be separate. Conversely, you would EITHER have it in interactive, lockdown, trust, or observation mode, so it seems it should be placed in the menu. So it makes more sense to put it there for that reasoning, but it also makes more sense because it would place observation mode right next to trust mode. As I said before, it wasn't clear to me that the trust modes would actually install permanent trust in anything run while they were active (I assumed they were what observational mode is). Putting the two modes next to each other will at least help some in making that more clear. It might also help to change the wording of the trust modes, something like "Trust All (Learning Mode) for..." or "Learning Mode (Trust All) for..." Using the term "learning mode" would make it more clear you're actually teaching SAP, and therefore what it "learns" will be permanent, vs just saying "trust" which doesn't imply permanence, and can be mistaken for merely observing/ignoring, which is what I did.

9. My point regarding restricted applications was simply that it seems counter-intuitive to have such a section (related to restriction) included under whitelisting settings. Not saying that's necessarily wrong, just that it means other sections, such as "Script," can therefore not be assumed to be whitelisted, since clearly not everything under "Whitelisting...Settings" is actually whitelisting related. And, as mentioned before, descriptions for various tabs and settings are often lacking (or, more often, simply absent altogether), and so there's no way of knowing how to interpret the contents of the script tab.

In any case, I still don't understand how the entries in the script tab are used since, as I said, I removed regedit as a test but was still able to both launch regedit.exe and run .reg files.

[hendy]

4. For Cancel button, for current UI design, the space is quite limited for adding an additional button. We will keep this in mind when we redesign the dialog.

5. The scan settings for APEX is actually shared with other offline engine, like ClamAV, or Avira (for users who opt to purchase Avira add-on).
    It is make sense to add additional text to make it clearer, for example UAV Scan Settings.
    APEX is not only for real-time, you can also use it for manual scanning (right-click, scan). APEX will be still active for manual scanning, when you disable the real-time scanning.
 

8. In observation mode, untrusted application will remain untrusted, and it will be blocked if it run when observation mode is switched off (https://support.secureaplus.com/what-is-the-observation-mode/).

There is some explanation about script at the following link: https://support.secureaplus.com/how-do-i-choose-to-associate-file-script-extension-types-to-script-interpreter/
basically if you defined in the script, if the file is not trusted, it will get blocked. For example if you add regedit.exe with .reg extension, it means that all .reg files opened by regedit has to be trusted. If you remove it, that means it will open all the .reg files, even if they are not trusted.
 

[S]

5. I didn't install it with the offline installer, since ClamAV is completely useless, so that must be why it won't do offline scans. I completely overlooked that fact even though I'm the one that installed it and I was aware there were two different versions, one with and one without offline AV, so I can only imagine the confusion for someone that is running it without having installed it (e.g. if I were to put it on my parents' computers, which is the goal for whatever setup I end up choosing), and so is just more reason to improve descriptions. So not only should it explain what APEX is and how it differs from UAV, but it should recognize whether or not an offline scanner (ClamAV, Avira) is installed and adjust accordingly in its description of its functionality and limitations (i.e. it should, among other things, explain that it is nonfunctional or limited in functionality, whichever is the case, when an offline scanner isn't installed). In any case, if I'm understanding correctly, APEX does nothing for me since I didn't install ClamAV.

8. I understand how observation mode works, my point was that I assumed trust mode worked that way. That is, I thought trust mode was what observation mode is. Since there is no observation mode accessible through the tray icon's menu, only trust mode, I thought that's what it was. And it doesn't help that the wording is misleading, e.g. "Trust all for x minutes" sounds like that's how long the trust will last, after which it will be revoked. Even the wording in the main app ("All trusted and untrusted programs are allowed to run temporarily") is misleading and makes it sound like the trust is only temporary. That's why I think observation mode should be added (aside from obviously just making it easier to access), because it would allow the user to see both simultaneously and realize there's a difference. Of course, that still wouldn't necessarily ensure they understand the difference, and a basic user (such as my parents if I were to set them up with SAP) wouldn't know, so personally I think in addition to adding observation mode to the whitelisting submenu, there should be some sort of labels for each, something like this:

Interactive Mode (Ask about untrusted) -- description not needed as much but doesn't hurt
Lockdown Mode (Deny all untrusted) -- description not needed as much but doesn't hurt
---------------------------------------
Trust Mode (Assign permanent trust)
  Enable for 5 minutes
  Enable for 30 minutes
  Enable until computer is restarted
---------------------------------------
Observation Mode (Temporarily allow without trusting) -- or just "Run without trusting" if you want to keep it shorter
  Enable for 5 minutes
  Enable for 30 minutes
  Enable until computer is restarted
 
9. So if I'm understanding you correctly, the scripts section then really has nothing to do with whitelisting, but instead indicates simply that they are monitored by SAP. So if they're listed on that tab, SAP will ask about or block untrusted scripts and allow trusted ones, but if they're not listed it will ignore/allow them. So this is clearly another area where a description is badly needed, as well as perhaps changing section labels, since this really doesn't seem to fit under whitelisting at all. Also, the linked page doesn't really make sense. It refers to trust level for the scripts (which makes sense, since they're files that will be either trusted or untrusted) and for the interpreters (which doesn't make sense, since there's no trust level indicated anywhere for them). I assume it means that if an interpreter is included on the script tab it's "untrusted," since that means SAP will intervene in the case of untrusted scripts, and if it's not included it's trusted, since SAP will ignore it, but that just doesn't sound right. Also, if that's the case, according to that page, it uses the *higher* level of trust, which also seems backwards, as it seems it should use the *lower* level of trust between the two. Though I suppose that could make sense, since if the interpreter is listed/untrusted, a trusted script would run because it's assumed safe since it's trusted, and if the interpreter is not listed and is therefore "trusted," it's assumed any script for it is ok, and so they're all allowed to run. So even though it seems backwards, it does make sense, assuming I'm understanding things correctly, but again it could certainly be explained better.

11. I think I've found another problem with SAP. As mentioned, I don't have internet right now, and that seems to be causing issues (at least I assume it's the cause, since I didn't experience these problems when I had internet). First, trying to run an untrusted program (installer) caused it to show the spinning circle by the cursor for a long time (initially it sat like that for at least a minute or two, then probably 10-20 seconds, then it started responding much quicker, within a few seconds) before the program finally attempts to run and SAP pops up. There was no difference when in observation mode. Once I trust the program, it runs right away, or at least within a few seconds, but I don't know if that's due to it being trusted or because it was getting better over time. Also, once trusted, if I try to set it as untrusted, the circle spins next to the cursor again, but this time most or all of the computer stops responding to the mouse (sometimes there's no reaction when hovering over desktop icons but there is when hovering over taskbar icons, and sometimes neither react to the cursor) and when I try clicking on the desktop the cursor goes away and it only shows the spinning circle, then after a while it goes white (like when a program stops responding) and then the desktop refreshes and things go back to normal, but the file is still trusted. The only way to get it to switch to untrusted is to wait and not do anything, but then it takes probably 30-60 seconds. I restarted the VM, then tried again, and it took almost exactly a minute and a half for SAP to pop up when I tried running the program. I blocked it then tried again, and the second time took almost exactly one minute, the next took ~55 seconds, then ~19, then immediately from that point on. Another interesting thing was that the first time or two that it was immediate, I got the SAP popup and chose to continue blocking, then the next two times I didn't get that popup and only got the error box that shows after blocking via the popup, then I got the popup again. Then I waited a bit (10-15 seconds) and ran the program again, and it took ~25 seconds that time. So it's all over the place. It shouldn't add a delay, and certainly not more than maybe ~5 seconds, just because it can't contact the cloud scanners.

[hendy]

Thank you very much for the explanation. Now we can understand better the point of view from your angle. We will try to improve the description in the future.

5. We have tried to install the version without ClamAV to replicate the issue, but so far APEX works in the offline scan (right click, scan).
    Probably what happen to you is the file is something that detected as clean by APEX. To test, you can download the following: http://2016.eicar.org/download/eicar_com.zip
    Extract the zip file, and you will get eicar.com file. Right click, scan eicar.com file, and you will see that APEX will detect the file.
    For the file that APEX did not detect, but you think that it is highly suspected as a virus, to help us to improve the product in the future, please submit the sample file to: https://secureaplus.secureage.com/Main/submit_malware.php

[S]

You're right. The context-menu scan (APEX) does work when offline. I think it was the system scan I was referring to, so I was getting the two mixed up again. That was is UAC, which is cloud-based, so won't work, but APEX does appear to work fine without internet. So that was my mistake.

I am finding that after getting the error 12007 (the server name or address could not be resolved) after having no internet for a while, when restoring it the error stays and SAP doesn't pick up on the fact there's a connection and resume/restart the scan and clear the error. And there's no way I see to manually tell it to retry, either. I even tried doing an update check, which requires internet access, and despite it successfully performing that it still doesn't register the fact it has internet and allow the UAV scanner to work. I had to reboot the VM to get it working again.

As for the malware file not being caught, I don't have the time right now to try with the eicar one you linked, but I've been testing with the infected ccleaner file downloaded from https://downzen .com/en/windows/ccleaner/download/5336162/. If I install SAP, then move the file onto the VM, SAP detects it. If I ignore that and trust the file, SAP continues to detect it. But if I put the file on the VM, THEN install SAP, when SAP installs it auto-trusts all files present, and never detects it as a potential threat. So my point was that SAP should ask during installation about auto-trusting files (and allow the user to select what directories should be auto-trusted and which ones shouldn't) and/or it should still scan trusted files at least the first time and alert if they're detected as malware. Because while I understand the premise of auto-trusting everything, since the assumption is being made that the system is clean when SAP is installed, that's not always going to be true, especially in a case like mine where I have the OS and all my data on separate drives, and it can lead to malware or questionable software previously downloaded being ignored because SAP trusted it, even though the user didn't.

I've also found a couple other issues. First, when clicking and dragging the windows from the minimize/close buttons (that is, mouse-down on one of those buttons and hold then drag, without releasing the mouse button), the window jumps so the mouse is holding on to the title bar more centrally. Granted, not a big issue, and not something likely to be experienced very often, but when it happens it's jarring, and it's certainly not normal behavior. Doing this on any other program simply does nothing. The second issue is that selecting multiple files and right-clicking then going to trust level should ideally show if they're trusted apps, trusted installers, or not trusted as long as they're all the same. This is more likely a design decision rather than a bug but, again, expected behavior, not to mention preferred and more helpful, would be to show their status the same as it does when only one is selected.