I need a way to analyze retail shelf photos, detect the shelf rail/edge angle relative to a horizontal line, and return “Not Detectable” if no valid shelf line is found. If the detected deviation is more than 15 degrees from horizontal, it should be flagged as misaligned.

The reference photos show shelf rails, price strips, and product rows that could be used as horizontal cues.

What I need advice on

• The most reliable way to detect shelf horizontal lines in a mobile app.

• How to calculate the angle of detected lines from horizontal.

• How to decide when the shelf is not detectable.

• Best practices for handling blurry, occluded, or low-contrast shelf images.

I attempted to use OpenCV with the Probabilistic Hough Line Transform and Apple's VNDetectHorizonRequest.**

However, the results have not been accurate or reliable.** The main issues I'm facing:

• Hough lines picks up too many non-shelf lines (product edges, label text, reflections)

• Results vary significantly with small changes in lighting or product density

• Cannot confidently determine when there are truly "no detectable" shelf lines vs. just noisy detection

• If Shelves are really angled, it gives incorrect results

What I'm looking for:

1. Is there a better pre-processing pipeline (morphological ops, masking, etc.) to isolate shelf rails specifically before running Hough?

2. Is there a scoring/confidence threshold strategy to decide when to return "Not Detectable" vs. a low-confidence angle?

3. Are there alternative approaches on iOS (CoreML, ARKit planes, CoreMotion) that would give more consistent results for this specific use case?

   I want users to avoid uploading images like these (cannot see products clearly from angles, shelves not detectable):

   
   [!'Example 1']