I think the answer is extremely simple. These Pharisees were like someone who saw a gnat or perhaps thought they saw a gnat in their bowl of pottage and they would get a strainer and work the soup over until they either had the gnat or convinced themselves that their soup was not contaminated. However, they would ignore something as significant as an entire camel in their soup and swallow it whole.
The reason it says "strain at" instead of "strain out" is because Jesus is pointing out that the gnat may not have even existed. They may have imagined it. By this, He showed that they were meticulous to such a degree on some points that they went out of their way to remove the supposed uncleanness whether it was really there or not. However, at other times, they completely allowed very important things to remain unchallenged. This is described in the previous verse, Matthew 23:23 - "Woe unto you, scribes and Pharisees, hypocrites! for ye pay tithe of mint and anise and cummin, and have omitted the weightier matters of the law, judgment, mercy, and faith: these ought ye to have done, and not to leave the other undone."
