Oil leak at crankcase bolt thread?

Hi - first noticed an oil leak at the water pump weep hole and replaced the water pump and oil seal. While I was looking for the oil leak I noticed some silicone on the upper most bolt that holds the crankcase together(the one that is infront of the flywheel cover...) I have just done a few hundred KM and notice oil on the crankcase, but its not coming from the weep hole....and it seems to be coming from the thread on the bolt....my question is can I put some plastic steel in the hole to stop the oil leak? The leak is very slow and does not drip off the bike....