Okay, after your explanation, I see exactly what you're doing. We had a guy a few years ago doing something very similar. He wanted to limit the forward position of the trigger or reduce the pretravel, and that's what you're doing, you're reducing the pretravel which sets the lip of the center blade up inside the frame meaning the drop safety blade is no longer effective.
So, your set screw has NO contact with the center trigger blade, absolutely none. What you're seeing happening with the drop safety blade is, with the screw backed out, limiting the forward position of the trigger, the drop safety is simply still captured up inside the frame and has not been able to reset to a location outside the frame. In addition, the more you unscrew that screw, positioning the trigger farther to the rear, the tab on the trigger bar is coming into contact with the ramp on the Firing Pin Safety (we normally refer to it as a Firing Pin Block FPB), and it's starting to push the FPB up into the slide. ALL of this is a bad idea, making the pistol unsafe, by bypassing the drop safety in the trigger and partially compressing the FPB into the slide.
My advice is to remove that set screw and wait for Apex to provide a trigger for the SF.
For educational purposes, remove your mag, check the chamber to make sure its empty. Now, shine a flashlight down the magwell and watch what happens as you move the trigger back and forth. Adjust the set screw to where the drop safety remains captured in the frame and once again, look down the magwell as you fiddle with the trigger. How much is the FPB compressed with the screw adjusted out, limiting the forward position of the trigger?
Here's a link to the udder guy's post.
https://www.waltherforums.com/forum/ppq/88233-mod-shorten-pretravel-trigger.html
Added: I didn't need the last picture. After I read your description in post 109, I had it figured out.