Tuesday, July 26, 2016

0patching Foxit Reader's Heap Buffer Overflow Vulnerability CVE-2016-3740

By Jure Skofic, 0patch team.

Working on a 0patch for Foxit Reader takes me back. One of the first patches we made was for Foxit and 0patch has come a long way since. We have a snappy agent which works reliably and we have the 0patch Factory which helps us develop and debug the patches. A lot of work has gone into making this happen and we'd like to share our know-how on writing patches.

In this blog post I'm going to describe in detail how to create a 0patch for a heap buffer overflow in Foxit Reader (CVE-2016-3740) using the official patch as a guideline. The vulnerability was reported to Foxit by Source Incite's Steven Seeley working with Trend Micro's Zero Day Initiative and was fixed with the release of Foxit reader 8.0. A proof-of-concept exploit was posted on Source Incite's GitHub.

The following tools were used to analyze the vulnerability and write the patch:
  • WinDbg
  • IDA Pro 6.9
  • Zynamics BinDiff 4.2
  • 010 Editor
  • 0patch Factory
  • 0patch Agent

The first step was reproducing the crash. The PoC is a TIFF image which Foxit reader converts into a PDF when opened. Here's WinDbg's output:

(2468.285c): Access violation - code c0000005 (first chance) First chance exceptions are reported before any exception handling. This exception may be expected and handled. ConvertToPDF_x86!CreateFXPDFConvertor+0x2da244: 5cf04d34 f3ab            rep stos dword ptr es:[edi]

The crash occurs in the ConvertToPdf_x86.dll library and is caused when rep stos tries to write to an unmapped memory section pointed to by edi. This causes the access violation. 
Next I disassembled the function where the crash occurred. It appeared to be an elaborate memset which tries to fill a buffer of a specified length, with a supplied character in 4 byte chunks. The pointer to the buffer, the fill character and the buffer length are passed to this function as arguments.

At this point I decided to make my life easier and performed a diff between the vulnerable and the patched version of the library. Since the library was heavily modified by the update, I couldn't rely on the diff to find the official patch. What I could do was follow the call stack and see if I end up in one of the modified functions.

Figure 1: Call stack at the time of the crash

The function the crash occurred in wasn't modified by the update, so I moved to the next one up the call stack. It appeared to be a wrapper for the previous function and wasn't modified either. The next function as seen in Figure 2, was the first one BinDiff marked as modified. What piqued my interest was the added branch resulting from a compare instruction:

Figure 2: Diff between the unpatched and the patched code

You'll notice that an additional check is performed where eax and ecx are compared. As marked by the green box on Figure 3, if ecx is larger or equal than eax, the execution continues on the left branch. If that's not the case, the execution follows the right branch and we enter the new code introduced by the official patch.

Figure 3: Official patch code

If you follow the right branch you'll notice something interesting. Amongst other stuff, a pointer to a string buffer is pushed onto the stack. Notice the //aNotEnoughMem_0 marked by the orange box on Figure 3? Disassembling the function gave me more reason to believe I was on the right track. After determining ecx is smaller than eax, a log function is called. One of the arguments passed to the function is a pointer to the string "Not enough memory at buf %lu (short %llu pixels)". After the log function call, the return value is set to 0 (xor eax, eax) and the patched function exits. This was solid evidence that a buffer length check was added in the patched version of the library. 

I needed to figure out if the added check was actually connected to the PoC. I used the 010 Editor to open the PoC. 010 Editor provides templates for a wide variety of binaries including TIFFs. The PoC description in Source Incite's advisory states that a large SamplesPerPixel value is used to cause the crash. Once I examined the PoC there was indeed a large value assigned to SamplesPerPixel - specifically 51201 as shown in Figure 5.

Figure 5: PoC analysis with 010 Editor

In order to determine if there was any connection between the PoC and the presumed patch code, I installed the patched version of Foxit Reader. I placed a breakpoint at the cmp ecx, eax instruction and ran the PoC to get the value of both registers. ecx had the value of 0xa68 (2664 decimal) and eax had the value 0x104294d (17049933 decimal). At first glance none of those values had anything to do with the large SamplesPerPixel value. I used 010 Editor to decrement the SamplesPerPixel value by 1 and ran the PoC again to see if any of the registers would change. I noticed the ecx value changed to 0x1042800. I decremented the value again and eax changed to 0x10426B3. For every decrement the value of eax was 0x14d less the the previous time. Next I divided the initial SamplesPerPixel value by 0x14d. The result was 0xC801 or 51201 decimal which is the exact SamplesPerPixel value of the PoC. At this point I was certain I had found the official patch for this vulnerability.

The next step was analyzing and applying the same patch logic to the vulnerable library. If we take a look at Figure 6 we can see that in both versions of the function, eax holds the same value - SamplesPerPixel value multiplied by 0x14d.  As indicated by the orange box on the patched execution graph in Figure 6, edi holds the pointer to an object passed as one of the function's arguments.  The vulnerable buffer is a member of this object, and the pointer to it is retrieved in ebx (mov ebx, [edi+0x240]). Next the buffer length is moved to ecx as indicated by the red box in Figure 6.

Figure 6: Diff between the vulnerable and the patched code

So to write the 0patch I needed to store the buffer length into ecx, compare ecx and eax and if eax is larger exit the function. Here's the patch pseudo code:

   ecx = [ebx+10h]; //Store the allocated buffer length in ecx
   if(ecx < eax)    //if ecx is smaller than SamplesPerPixel 
   {                //value multiplied by 0x14d, exit the 
      return 0;     //function
   else continue

The optimal spot for patching would be 0x5ce994ce, right after we get the object pointer in ebx, so the correct value can be copied to ecx. Here's the 0patch code:

  mov ecx, [ebx+10h] ;store the allocated buffer length in ecx
  cmp ecx, eax       ;compare to the length of input data
  jge end_patch      ;if ecx is greater or equal jump to 
;end_patch tag and continue
   xor eax, eax       ;else if ecx is smaller
  pop edi
  pop ebx
  retn               ;return 0

Now that I had the patch code and offset, it was time to create a 0patch using 0patch Factory. It's an awesome tool built by my colleague Luka Treiber which makes building 0patches much easier. It takes the 0patch source code and builds it into a 0patch ready for deployment with our agent. It also provides nifty tools for debugging the patches. I used one of 0patchFactory's templates and created the 0patch source code.

MODULE_PATH "C:\Program Files (x86)\Foxit Software\Foxit Reader
\plugins\Creator\x86\ConvertToPDF_x86.dll" PATCH_ID 249 PATCH_FORMAT_VER 2 VULN_ID 1441 PLATFORM win32 patchlet_start PATCHLET_ID 1 PATCHLET_TYPE 2 PATCHLET_OFFSET 0x002794CE JUMPOVERBYTES 0 N_ORIGINALBYTES 5 code_start mov ecx, [ebx+10h] cmp ecx, eax jge end_patch call PIT_ExploitBlocked xor eax, eax pop edi pop ebx leave retn end_patch: code_end patchlet_end

Mitja already explained in detail how to build and deploy 0patches (see Writing a 0patch for Acrobat Reader's Use-After-Free Vulnerability CVE-2016-1077), so I won't go into that here. 

When I tested the 0patch, Foxit Reader displayed an error message. This behaviour is exactly the same as with the official patch. We added the Exploit Attempt Blocked dialogue for added measure, to alert the user about the attack.

That's about it. The 0patch contains 8 instructions and took me a day to analyze and develop it.

If you already have 0patch Agent installed, the patch (ZP-249) should already be deployed on your machine - all you need to do is install the vulnerable version of Foxit Reader and open the PoC with it. In case you're new to 0patch, simply create a free 0patch account and install 0patch agent

This is a great example how 0patch could be used in bridging the security update gap. Foxit's official update contains a lot of new code. According to BinDiff about 10% of ConvertToPdf_x86.dll's code was added or modified. The update addressed multiple vulnerabilities and possibly added some functionality. That's a lot of new code to test compared to the few instructions in a 0patch. Less code, less testing.

For anyone interested in writing their own 0patches, please contact us at crowdpatching@0patch.com.

Jure Skofic


Monday, June 20, 2016

New Release: 0patch Agent 2016.06.14.850

We released a new build of 0patch Agent for Windows today. There are two reasons for this: we'd like to test the updating mechanism with our beta users; and we have fixed a couple of issues reported by our beta users, plus a few minor issues we had on our own hit list.

The two issues reported by our beta users were:

  1. Gjoko Krstic of Zero Science Lab reported a security issue in the way 0patch Service was registered: we forgot to enclose the path to the service in quotes when registering it, which allowed a local attacker with ability to create an executable (program.exe) in the root of the drive where 0patch Agent was installed to have this executable launched as user SYSTEM whenever 0patch Service was launched (e.g., after restarting the computer or updating the agent). You can find more information about this type of vulnerability here and here. After being alerted of this issue, we reviewed the entire code for similar issues and indeed found two in the way 0patch Tray was executed, both from the installer and upon login.
    It may be tempting to conclude with the impression that default Windows permissions do not allow regular users (those that don't already have admin privileges in some way or another) to create files in C:\ root, which would effectively make this a non-issue. But that would not cover all cases: one could still install 0patch Agent on a non-default location, e.g., D:\Apps\My apps\, and there is no guarantee that a non-admin user couldn't create D:\Apps\My.exe there.
  2. Ryan O'Horo alerted us about a number of silly terms in our beta license agreement, which we forgot to update before going public. While we responded to his alert with an updated agreement on our web site, we have now also put this updated agreement in the installer.

Big thanks to Gjoko and Ryan for providing valuable feedback and helping us improve. And they're not the only ones: we're getting great feedback from many other beta users as well, including some issues related to user interface, which we'll address at a later time. Let us reiterate: You guys are the real value of this beta! Thank you!

When your 0patch Agent syncs (which it does once an hour, but you can also force it through 0patch Console), it will detect a new version of itself being available and will inform you about it. All you need to do is open the 0patch Console and click "Get latest version" as shown on the image below. The console will disappear and relaunch after a while, showing the new agent version (2016.06.14.850).

We'll appreciate your telling us about any problems encountered during this update process or elsewhere. Just shoot us a quick email to support@0patch.com (or encrypted with our PGP key to security@0patch.com). Thanks!

Friday, June 17, 2016

Writing a 0patch for Acrobat Reader's Use-After-Free Vulnerability CVE-2016-1077

This blog post will describe how we created a 0patch for CVE-2016-1077, a use-after-free vulnerability in Adobe's Acrobat Reader DC version 15.010.20060 for Windows.

According to Adobe Security Bulletin APSB16-14, this vulnerability was reported to Adobe by Pier-Luc Maltais of COSIG, and shortly after the release of this bulletin, a proof-of-concept exploit was published on Packet Storm. This PoC is in the form of a PDF document that crashes the Reader when opened. (By the way, it was apparently created with single-byte raw file fuzzing; we were able to compare it to the original PDF file we found on the Internet.)

This was enough for us to start analyzing the vulnerability. To keep this blog post reasonably short, we will omit the vulnerability analysis process, which is nicely described in this detailed post by Fortinet's Kai Lu and Kushal Arvind Shah for another identical vulnerability in the very same function as ours.

So let's fast-forward to the point where we know exactly what the flaw is. We have a use-after-free issue, as shown in the vulnerable code below.

Vulnerable code: after freeing an object, reference to this object is not removed

The above image shows three relevant code blocks somewhere in Acrobat Reader's implementation of deflate operation on an XObject (an element in a PDF document), where the code is obviously doing some clean-up by freeing an object it no longer needs. The pointer to this object is retrieved in register ecx, then the test ecx, ecx instruction effectively checks if the pointer is NULL, and this determines whether the execution will continue on the red branch (ecx not NULL) or the green branch (ecx is NULL). In the red branch, pointer to object's free function is copied from object's vtable (free is the first function in vtable in this case) to register eax, then a call to free is made.

The relevant code can be represented by this simple pseudo-code:

ObjectA* a; // stored in edi register
ObjectB* b; // stored in eax register
ObjectC* c; // stored in ecx register
b = a->some_objectB;
c = b->some_objectC;
if (c) // the red branch
  delete c;

Note that after c is deleted, its reference (pointer to it) in b->some_objectC is not removed. And sure enough, at some point later in the execution, b->some_objectC  is accessed again. But this time c points to an already deallocated memory address, and access violation occurs.

Now let's see how we would fix the above pseudo-code. It's trivial, we only need to write NULL to b->some_objectC after deleting c:

b = a->some_objectB;
c = b->some_objectC;
if (c)
  delete c;
b->some_objectC = NULL; // destroy reference to deleted object

So we have a pseudo-code patch, now let's translate it to machine code. First look at the vulnerable code in a "text" view:

6056E274 8B 47 38           mov   eax, [edi+38h]
6056E277 C6 45 FC 0B        mov   byte ptr [ebp+var_4], 0Bh
6056E27B 89 88 C8 00 00 00  mov   [eax+0C8h], ecx
6056E281 8B 4F 3C           mov   ecx, [edi+3Ch]
6056E284 E8 46 07 00 00     call  sub_6056E9CF
6056E289 8B 47 3C           mov   eax, [edi+3Ch]  ; edi points to some object, eax is pointer to

                                                    a member object
6056E28C 8B 88 C8 00 00 00  mov   ecx, [eax+0C8h] ; ecx is pointer to an object member from the

                                                    "eax" object
6056E292 85 C9              test  ecx, ecx        ; Is this pointer NULL?
6056E294 74 05              jz    short loc_6056E29B ; If it's NULL, we don't have to free it

6056E296 8B 01              mov   eax, [ecx]      ; Get the address of function "free" from
                                                    object's vtable
6056E298 56                 push  esi             ; esi = 1
6056E299 FF 10              call  dword ptr [eax] ; call "free" for this object

6056E29B    loc_6056E29B:                           ; CODE XREF: sub_6056DC50+644 j
6056E29B 56                 push  esi             ; the reference to freed object is not removed

6056E29C 6A 24              push  24h
6056E29E E8 71 B6 AD FF     call  sub_60049914

6056E2A3 59                 pop   ecx
6056E2A4 59                 pop   ecx

The patch code must come after call dword ptr [eax], which is a call to object's free. We need the address where the pointer to the object is stored, so we can put a NULL there.This address was [edi+3Ch]+C8h just a few instructions earlier, and we know edi is not corrupted by the call to free as it is still being used later in the code without being assigned a new value. (This is easy to check with IDA as it highlights all uses of a register when you click on one.)

When creating a micropatch, we need to be careful about a few things:

  1. Injecting a micropatch works by overwriting one or more existing instructions with a 5-byte JMP instruction to some nearby-allocated memory block, where the overwritten instructions are added at the end of our patch code, followed by a JMP back to the place in the original code right after them. Of the overwritten instructions, the first one may be a destination point of some CALL or JMP, but others may not be, as that would certainly lead to an error in execution. Fortunately, IDA makes it really easy to verify this with its graphical representation of code.  
  2. We should not cause any unwanted side-effects, like corrupting values of registers or stack-based variables that subsequent code may still be using. When in doubt, we should PUSH/POP the registers we use or otherwise preserve them.
  3. We should not merely assume that a register value will survive a function call (e.g., that edi will survive a call to free in our case) - we must prove it by either reviewing this function (and all functions it calls etc.) and showing that the value is reliably preserved, or showing that the original code after the function call relies on the fact that the register hasn't changed.
  4. Our code should be as easy as possible to review. On one hand, this means using as few instructions as possible, and on the other hand it means making it easy for a reviewer to verify our claims upon which the patch code is constructed (e.g., that "patch code can safely change eax because...", or "at this point, edx will always point to..."). Sometimes, these goals are in contradiction: for example, in our patch code below, we could simply use eax instead of edi and avoid having to PUSH/POP edi, but it would make code review harder.

First, let's find a suitable location for our patch. There's an obvious candidate immediately after  call dword ptr [eax] at location 6056E29B (AcroRd32.dll + 0x56E29B). The instructions there are:

6056E29B 56                 push  esi
6056E29C 6A 24              push  24h
6056E29E E8 71 B6 AD FF     call  sub_60049914

Remember, we need 5 bytes for our "JMP to patch", so we'll have to relocate all these three instructions to the space right after our patch code. Note that the third instruction is a relative CALL, which means that its offset has to be recalculated when it is moved. (Don't worry, 0patch agent does this automatically.)

As for the patch code, let's try this:

push edi                   ; store edi as subsequent code still needs it
mov edi, dword[edi+03ch]   ; edi = pointer to object b in pseudo code
mov dword[edi+0c8h], 0h    ; set pointer to object c to NULL
pop edi                    ; restore edi

This code just uses a single register (edi), making it trivial for a reviewer to see that it has no side effects while writing a NULL where the original code forgot to do it. As already noted, we could make it even shorter if we used eax instead of edi, as we wouldn't have to preserve eax's value. However, proving that we don't have to preserve eax's value would require reviewing quite some additional code after our patch location, including a function that is called there. This would consume patch developer's as well as patch reviewer's time, and increase the risk of error for both of them.

One last check: Can we be absolutely sure that edi will point to the same object a at address 6056E29B as it did a couple of instructions back at address 6056E289? There are two execution paths between these two addresses (as IDA nicely shows). One goes through the aforementioned green branch, and the other goes through the red branch. The "green branch" path is easy to prove: there is simply no instruction there that would change edi, so it must still have the same value at the end. The "red branch" path, however, includes a call to free from object's vtable. Proving that this function does not alter edi could be a daunting task, especially as IDA can't show you where the code behind this dynamically-set free is. (In fact, this function could be different for various types of objects being processed here, and only the original developers of this code know what objects that could possibly be.) So we'll use another approach: we look at the code after our patch location to see whether it relies on edi being the same as before. And in fact, we can see that it does (but that's beyond the scope of this article).

We have all we need, but how do we turn this into an actual 0patch? We use 0patch Factory. 0patch Factory is a simple tool we wrote, which takes a 0patch source file as input, and turns it into a 0patch that can be immediately applied on any computer with our 0patch Agent. Let's take a look at a 0patch source file for this micropatch:

MODULE_PATH "C:\Program Files (x86)\Adobe\Acrobat Reader DC\Reader\AcroRd32.dll"
VULN_ID 1272




    push edi
    mov edi, dword[edi+03ch]
    mov dword[edi+0c8h], 0h
    pop edi


Some explanation is probably in order:

  • Each 0patch consists of one or more patchlets; a patchlet is a set of machine code instructions that are to be injected at a specific offset from the module's base.
  • MODULE_PATH is full path to the module (binary file) we're patching. 0patch Factory needs this for two reasons: (1) calculating the module's hash, as the patch should only be applied to this exact binary, and (2) extracting the original bytes from all locations where we're injecting our patchlets, as any patchlet should only be applied when the bytes found in the "live" module on its injection location exactly match these original bytes.
  • PATCH_ID is a unique ID of the patch in the 0patch database. It can be arbitrary during patch development, but a patch will get a suitable unique ID before it gets signed and uploaded to the server.
  • PATCH_FORMAT_VER is the version of the format in which the patch is written. 2 is the only supported value at this time.  
  • VULN_ID is the ID of the vulnerability (in the 0patch database) fixed by this patch.
  • PLATFORM is either Win32 or Win64 at this time, specifying the format of machine code in this patch's patchlets. Acrobat Reader DC we're patching here is a 32-bit application.
  • patchlet_start and patchlet_end mark beginning and end of an individual patchlet.
  • PATCHLET_ID is a unique ID of a patchlet inside a patch, usually iterative from 1 upwards.
  • PATCHLET_TYPE is the type of the patchlet, currently the only supported type is type 2 (which means "code injection patchlet"). In the future, we'll support other types of patchlets.
  • PATCHLET_OFFSET is the offset from the module's base where the patchlet code should be injected. In our case, this offset is 0x56e29b.
  • N_ORIGINALBYTES is the number of original bytes that we should check when applying the patchlet. The default is 5, which means all 5 bytes that we will overwrite with the "JMP to patch" instruction.
  • code_start and code_end mark beginning and end of patchlet's code, which will be compiled to binary code by 0patch Factory.

We store this 0patch source file in ZP-245.0pp and build the patch using 0patch Factory on a computer with 0patch Agent installed:

0patch Factory makes it easy to build a micropatch: just
right-click a 0pp file and select "Build Patch"

This both compiles and installs the new 0patch, making it ready for immediate application to a running vulnerable Acrobat Reader. Without further ado, let's open POC_ZP-245.pdf to test our patch.

0patch Agent notifies you that patch 245 just
got applied to the running Acrobat Reader

Acrobat Reader gets launched, the "Patch Applied" pop-up is shown notifying us that patch 245 was applied to  AcroRd32.exe, and... no crash. The document seems to be displayed correctly. We scroll down and get the "A drawing error occurred." error message from the Reader, but this is expected, as there is still a corrupt graphical element in the PDF document, and Reader correctly notifies us about that. (Latest version of Reader also shows this message.) We can continue to use Acrobat Reader as if nothing has happened.

There you go, we've just created a simple micropatch for a remotely executable memory corruption vulnerability. It consists of just four machine code instructions and can be instantaneously applied to a running Acrobat Reader while a user is reading some document with it.

Should you like to experiment with this micropatch on your own, create a free 0patch account and download 0patch Agent for Windows if you haven't already. (If you have, than patch ZP-245 has probably already been downloaded by your agent and is waiting for you to run the vulnerable Acrobat Reader.)

Patch 245 is now in the "Patches" list in the 0patch Console, where you can enable or disable it

Once you have 0patch Agent installed and registered, install Acrobat Reader DC 15.010.20060 and use it to open POC_ZP-245.pdf, first with patch 245 enabled, and then disabled. Notice the difference? ;)

If you'd like to write your own 0patches, and we sure hope many of you would, give us some time to polish our 0patch Factory for you. It's nothing fancy, but you deserve a decently tested tool with useful instructions. In the mean time, please send us an email at crowdpatching@0patch.com to express your interest. We look forward to our collaboration in changing the way vulnerabilities are getting fixed.


Tuesday, June 7, 2016

0patch Open Beta is Launched

After a long period of internal development and testing, our mighty little patching machine finally got wings and flew out of the nest.

This is BIG for us. We've invested a significant part of our last three years into building a technology and a business model that we believe can make a big difference in how vulnerabilities are getting fixed, and consequently make attacks considerably harder and more expensive. It is our sincere hope that the concept of vulnerability micropatching will some day become a "goes without saying" feature in all computers, from large data-processing machines and employee workstations, to car computers, mobile phones and the tiniest Internet-of-Things gadgets.

0patch does not claim to be a silver bullet, and it only aims to solve a very specific problem of patching vulnerabilities, but we believe it's the most efficient possible way to bridge the security update gap that makes it so unforgivably easy to break into any network today.

We're very excited about new users testing our technology, as well as security researchers getting a tool for fixing the vulnerabilities they find. However, as with any new technology, we're expecting that the wheels will sometimes get stuck, and things might crack or break in unexpected ways - that's the point of testing. So please remember that our technology is in beta, don't use it in production yet!

Getting Started

Enough of small talk. For an introduction to 0patch you can watch the DeepSec 2015 presentation or get the main idea from this compact 3-minute video.

To try out 0patch, simply head over to our web site, download the 0patch agent for Windows (sorry, Linux and Mac agents are still in the works) and install* it on some Windows machine. We support all Windows platforms from XP and Server 2003 onward.

Once you install the agent, you will be asked for email and password to register your agent on the server. Since you probably don't have an account yet, click "SIGN UP FOR A FREE ACCOUNT" and create an account on our server, then continue registering your agent. Once the agent has been registered, it will download all currently available micropatches from the server and immediately start applying them to newly launched and already running processes on your computer.

(Important: the agent must be able to connect to our server in order to download patches. If you're installing the agent behind a proxy and/or firewall, follow the steps described in user manual, section "Network Connectivity".)

However, you probably won't see anything interesting happening, because you don't have the exact vulnerable applications installed that we have patches for at this early stage of patch development. To really see how 0patch works, you'll have to install some of these applications and run proof-of-concept exploits against them. Luckily for you, we have prepared a couple of vulnerable applications' installers and their respective exploits, which you can find at https://0patch.com/poc/. Simply install one or more of these applications and launch their associated exploits (e.g., POC_ZP-3.pdf for Foxit Reader 4.1.1), once with 0patch agent enabled and once with the agent disabled, to see the difference.

You can change pop-up settings to get notified about each applied patch for full Hollywood effect :) But seriously, these pop-ups are really here just to demonstrate how 0patch works, and for you to help you test it. In actual production, users won't want to be bothered with these "details", and only admins will be interested in events generated by 0patch.

Please refer to the user manual  to learn how to navigate the 0patch agent console. If anything remains unclear or confusing, or if you encounter any problems with 0patch whatsoever - be it errors, strange behavior, negative impact on anything, or any crashes or freezes -, please let us know by sending an email at support@0patch.com or filling out a support form at https://0patch.zendesk.com.

We hope you'll be as excited about our technology as we are. Remember: the goal is to make attackers' lives much harder. Do you think this is the right way to go? Do let us know.

* (Big thanks to Ryan O'Horo for alerting us about silly draft boilerplate terms regarding marketing, funding, confidentiality and reverse engineering in our beta license agreement included with the first public 0patch Agent build. We corrected this in a subsequent build.)

But Where Are The Other Patches?

0patch is per se not a solution for vulnerability patching, it is merely a tool that allows micropatches to be inexpensively delivered to users and then quickly applied to new and running processes (or removed therefrom) without disturbing users in any way. To prove the concept, our team has so far created 200+ micropatches for various types of vulnerabilities in different software products, including Java Runtime, Foxit Reader, Adobe Reader, Firefox and Windows schannel and usp10 libraries.

However, in order to really make an impact on security, we need thousands of micropatches for existing vulnerabilities and also fresh "temporary" micropatches every time an official security update is published by a software vendor. Ideally, we'd like to see software vendors embrace the concept and start delivering micropatches for their vulnerabilities, but there will be a constant need for 3rd party micropatches for those that don't or can't.

Our team is far too small to create all these patches. In fact, we believe only a global crowdsourced community of security researchers can tackle this problem. And now that the technology is available, this is our next step: building a tribe of 0patch contributors who will augment their existing vulnerability research and reverse engineering skills with a new challenge, creating a micropatch. We call this crowdpatching.

Only a powerful, knowledgeable community with a sincere desire to make things better will be able to produce the amount of micropatches that will make users significantly more secure. Once the technology is adopted and traction is gained on the user side, it will also become possible to motivate researchers financially, as we all know that you can't keep a machine running solely on enthusiasm.

That being said, we have built a so-called 0patch Factory, a tool that makes it easy for security researchers to compile micropatches from source code into a form understood by the 0patch agent, and quickly test them. It still needs a few final touches before we can share it with researchers, but we'll appreciate your expressing interest in writing micropatches by sending us an email to crowdpatching@0patch.com. We're excited to hear what you think and help you create your first micropatch.

Security of 0patch

0patch is a security technology, i.e., its goal is to increase the security of a system. Our team has put a lot of effort into building a resilient architecture with a security model that doesn't explode in contact with today's reality (servers get compromised, communications are monitored by adversaries, malware may already be running on computers), and into code that hopefully shouldn't contain security bugs that we regularly find in other people's code.

Obviously, every additional functionality - even security functionality - brings new attack surface to the system, through which attackers gain new interaction points that they can try to abuse. 0patch is no exception: just the fact that our agent has ability to inject new code into any running process, and that this code comes from our server, must give pause to anyone concerned about security. (Never mind that every software updater presents essentially the same kind of security concern.)

Our security model is built on these premises:

  1. The server will get compromised, even though we harden it and promptly apply all security updates. When it does, the attacker should not be able to diminish the security of connecting users' systems, and the impact should be limited to denial of service for delivery of new micropatches. Gaining full control of the server and its database should not allow the attacker to start delivering his own code to users or prevent agents on users' computers from applying the micropatches they already have. We achieve this goal by digitally signing each patch, and each patch revocation, with a dedicated private key on an air-gapped machine, then uploading the signed patch blob to the server. The agent refuses to accept a patch or a revocation from the server without it being accompanied by a valid signature.
  2. Communication between agents and server can be monitored and actively modified by attackers. Again, this should not allow the attacker to either deliver malicious code to users or prevent agents on users' computers from applying existing micropatches. While all agent-server communication is done via securely configured HTTPS, we all know that certification authorities can get compromised. The problem of this attack scenario then becomes a subset of the"compromised server" scenario above.
  3. Malware will get executed on users' computers with 0patch agent installed. If such malware has system or administrative privileges, there is nothing we can do. However, malware with limited privileges should not be able to abuse agent's functionality or data to elevate its privileges. Furthermore, such malware should not be able to prevent 0patch agent from applying micropatches to processes on the computer (as it could then exploit an otherwise micropatched vulnerability in a privileged process).

It is tempting to think that our 15 years of finding all sorts of vulnerabilities in our customers' products and many widely-used applications, discovering new attack methods and improving existing ones would qualify us for developing a product without any significant vulnerabilities. But we're just people with blinds spots like everyone else, and we have missed things - perhaps big things. So we humbly submit our work to the global security research community to find bugs and hopefully tell us about them.

If you find a vulnerability in 0patch agent, server, or anywhere else, and would like to report it to us, please do so via security@0patch.com and preferably use our PGP key. We'll highly appreciate your effort and look forward to experiencing being on the receiving end of vulnerability disclosure. And just imagine: what would be a better validation of this technology than 0patch agent micropatching its own vulnerability?

The 0patch Team

In closing, a few words about the people behind 0patch. 0patch was built by a small group of security veterans who otherwise go by the name of ACROS Security, and a handful of external contributors. We were frustrated by the fact that with all the "advances" in information security, our penetration testing engagements, where we realistically simulate an actual attacker, have not gotten any harder in the 15 years of our practice. And this means that actual attackers' job also hasn't gotten any harder. So we made a decision and dedicated a significant chunk of our time to building something that, if implemented properly, would make our job much harder.

This is our team:

  1. Simon Raner - 0patch agent for Windows, writing patches
  2. Jure Skofic - 0patch distribution server, writing patches
  3. Renata Stupar - vulnerability management, business intelligence, testing
  4. Luka Treiber - 0patch factory, writing patches, internal security reviews
  5. Stanka Salamun - business development, project management  
  6. Bostjan Praznik - 0patch agent GUI for Windows
  7. Bozidar Jovanovic - branding and graphic design
  8. Robi Faric - 0patch agent for Mac and Linux
  9. Mitja Kolsek - security architecture, writing patches

We're proud to have been given this opportunity to make a serious global change, and believe that the time is ripe for it. We hope our contribution will give rise to many discussions, new ideas and constructive criticism, eventually leading to a better standard approach to fixing vulnerabilities.

We'd like to extend a warm thank you to our closed beta users who have invested a lot of their time in helping us analyze and fix bugs. We wouldn't have a decent product to share with the world without you!

And finally, immense thanks to all new beta users, security researchers and software vendors for your help, support and encouragement. We need to remind ourselves that we're all on the same team when it comes to defending our systems against attackers.We hope that 0patch will provide an additional link that will connect us in our joint efforts.


Friday, January 22, 2016

Bridging the "Security Update Gap" With 0patch

Vulnerability Patches Can be Really Small and Easy to Apply

Yesterday we tweeted a proof-of-concept actual micropatch for the "Winshock" vulnerability (CVE-2014-6321, MS14-066) in Windows schannel.dll. The patch fixes a buffer overflow vulnerability that allowed attackers to execute arbitrary code on any SSL-enabled IIS server. (Thanks to Mike Czumak, BeyondTrust and Malware Tech for their awesome analyses, especially Mike for also sharing his method for triggering the bug.)

Our "0patch" for Winshock consists of just 11 machine instructions (28 bytes), which is a fairly typical size for a 0patch judging from over 300 0patches we've written so far. To put these 28 bytes in perspective, you should know that the official Microsoft's update that fixed the same vulnerability (although it might also have changed some other bits here and there) was 243 KB, which makes our 0patch roughly 8,800 times smaller than the official fix.

You have to understand that vendor updates - be it Microsoft's, Adobe's, Oracle's - simply replace entire executable files, even if just a single sanity check has been added to them to fix a vulnerability. This has several unpleasant side-effects:

  1. In order to replace an executable file that is currently in use by one or more running processes (which is often the case, including with the above-mentioned vulnerability), these processes need to be terminated, which leads to something we all hate: restarting the computer.
  2. In the unlikely but possible event that the vendor update should cause functional problems, reverting the change (i.e., uninstalling the update) again requires a computer restart. In contrast to installing updates, which can be managed centrally for an entire enterprise, uninstalling requires manual handling of each affected computer - which is especially difficult with laptops traveling around the world.
  3. Large updates are impossible for administrators (or basically anyone) to thoroughly review before installing. What are the changes they introduce, do they also "silently" modify some functionality? 

As a result, enterprise administrators schedule extensive and lengthy testing of vendor updates before installing them in production. It is common to have vendor updates installed in an enterprise several months after they have been released. Think about it: attackers start reverse-engineering vendor updates the same day these get released, and often have reliable exploits in just a few days. Defenders, on the other hand, are struggling with testing these updates and remain vulnerable to exploits extracted from official vendor updates. By the time defenders have finally installed vendor updates from two months ago, they're already vulnerable to the exploits from last month's updates. The game seems to be eternally rigged in favor of attacker.

Now this problem is getting a solution. 0patch can help defenders bridge this "security update gap" by 0patches protecting them from newly extracted exploits while the official updates are being tested. Since creating a 0patch from an official update is a similar process to creating an exploit from the same update, defenders finally have a chance to compete with attackers - and even outrun them.

And why would administrators dare to apply 0patches to production without rigorous testing? Here's why:

  1. 0patches are designed to be applied to running processes. This means that the above-mentioned Winshock 0patch gets applied to a running IIS server without having to even restart a service, much less the entire computer.
  2. 0patches are designed to be removed from running processes just as easily - no restarts. This allows admins to instantly revert to pre-0patched state (i.e., instantly un-applying the Winshock 0patch from a server).
  3. Both applying and removing 0patches can be done remotely, from a central location.
  4. 0patches are so small that anyone with some knowledge of machine programming can review them. This means that administrators know exactly what changes they're introducing by applying 0patches.

Making the "security update gap" a thing of the past is one of our most important goals. Until we get there, no matter what security technology corporate and government networks are using, breaking into them will remain a child's play.

Subscribe at https://0patch.com and become part of the solution.


0patch: Fixing The Fixing

Microscopic Cures for BIG Security Problems

Those of you following our work at ACROS Security have noticed the near-silence in our public department during the last two years. The blog was static, there were no news on the web site to speak of, and googling us gave no recent hits. Sure, our customers know we were as busy as ever under the blanket of serial NDAs, but what was going on in our "free" time?

One word: 0patch. We were building what we believe is going to fundamentally change the security game. And it seems so trivial, it's hard to comprehend that something like this hasn't been a standard procedure for ages. Well, no-one seemed to have bothered building it*, so we did.

Critical remotely exploitable vulnerabilities have become a daily routine, but fixes for them are still sporadic and delayed if they exist at all, providing attackers with a growing supply of ammo. Believe us, it does get frustrating when you can use the same "pick an exploit, phish, win" procedure for 15 years and successfully break into every single network, despite all the fancy security technology they're using. Defenders have absolutely no fighting chance.

It became clear to us that whatever else our security industry does to protect against breaches will remain utterly futile until we fix the fixing.

Thus, 0patch (pronounced 'zero patch') got presented to the public for the first time at the DeepSec 2015 conference in Vienna (slides, video).

0patch is a platform for instantly distributing, applying and removing microscopic binary patches to/from running processes without having to restart these processes (much less reboot the entire computer).

0patch doesn't change a single byte on the file system: all patching is done in memory, as soon as a vulnerable module (e.g., EXE or DLL) is loaded by any process.

Patches deployed by 0patch (called '0patches') are extremely small, usually containing just a handful of machine instructions. This makes it easy to review them and absolutely minimizes the risk of them causing functional problems to the patched processes. Compared to typical official vendor updates that also just fix a couple of vulnerabilities, 0patches are roughly a million times smaller! And about a million times easier and faster to apply and remove.

0patch allows vulnerability researchers to create patches instead of exploits, and to get paid for that by the very consumers of these patches.

And finally, 0patch allows software vendors to fix vulnerabilities in their products running on users' computers quickly and cheaply, providing an unprecedented ability to actually outrun attackers for the first time in history.

You're welcome to learn more at https://0patch.com and follow us at @0patch.

Godspeed, 0patch!

* Granted, we're nowhere near the first to be doing hot-patching or unofficial vulnerability patching (shoutout to Determina and Alex Sotirov, eEye, Luis Miras, Ilfak Guilfanov,the PatchDroid team, Jeff Arnold and M. Frans Kaashoek and the Ksplice team, ZERT et al., and those whose ideas they have borrowed), but there is currently no generic production-grade solution allowing system administrators to instantly hot-apply official or unofficial micropatches with minimum risk of functional problems, and instantly hot-remove them should such problems happen to occur.