我正在为我的大学项目名称构建一个android应用程序"AI肺炎检测仪"；其从用户胆囊获取输入图像并预测该人患有肺炎的可能性。
在我的应用程序中，我导入了模型，从库中获取图像的位图然后将其调整为(224224(，这是模型的输入大小然后将其转换为张量图像然后从张量图像中获取字节缓冲区然后将字节缓冲区作为输入馈送到模型
但在所有这些模型之后，每个图像的值总是从0.0039开始(即使是在应该>0.5的正图像上((模型在python上运行得很好(。
我的应用程序代码是：
package com.shekhardwivedi.aipneumoniadetector;
import androidx.appcompat.app.AppCompatActivity;
import android.app.Activity;
import android.content.Intent;
import android.graphics.Bitmap;
import android.graphics.Matrix;
import android.net.Uri;
import android.os.Bundle;
import android.os.Debug;
import android.provider.MediaStore;
import android.view.View;
import android.widget.Button;
import android.widget.ImageView;
import android.widget.TextView;
import com.shekhardwivedi.aipneumoniadetector.ml.AiModel;
import com.shekhardwivedi.aipneumoniadetector.ml.Stacked;
import com.squareup.picasso.Picasso;
import org.tensorflow.lite.DataType;
import org.tensorflow.lite.support.common.ops.DequantizeOp;
import org.tensorflow.lite.support.common.ops.QuantizeOp;
import org.tensorflow.lite.support.image.ImageProcessor;
import org.tensorflow.lite.support.image.TensorImage;
import org.tensorflow.lite.support.image.ops.ResizeOp;
import org.tensorflow.lite.support.tensorbuffer.TensorBuffer;
import java.io.ByteArrayOutputStream;
import java.io.Console;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.nio.ByteBuffer;
public class predictionscreen extends AppCompatActivity {
Button upload;
TextView prediction;
ImageView imageView;
boolean debuFlag = false;
String debuGlobal = "";
public static final int GET_FROM_GALLERY = 3;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_predictionscreen);
upload = findViewById(R.id.button);
prediction = findViewById(R.id.textView);
imageView = findViewById(R.id.imageView);
prediction.setText("Result: UnavailablenUpload Image");
Picasso.get().load(R.drawable.tittle).into(imageView);
upload.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View v) {
startActivityForResult(new Intent(Intent.ACTION_PICK, android.provider.MediaStore.Images.Media.INTERNAL_CONTENT_URI), GET_FROM_GALLERY);
prediction.setText("Upload Image");
}
});
}
@Override
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
super.onActivityResult(requestCode, resultCode, data);
//Detects request codes
if(requestCode==GET_FROM_GALLERY && resultCode == Activity.RESULT_OK) {
Uri selectedImage = data.getData();
Bitmap bitmap = null;
try {
bitmap = MediaStore.Images.Media.getBitmap(this.getContentResolver(), selectedImage);
//Picasso.get().load(data).centerCrop().into(imageView);
imageView.setImageBitmap(bitmap);
int width = bitmap.getWidth();
int height = bitmap.getHeight();
int size = bitmap.getRowBytes() * bitmap.getHeight();
ByteBuffer imageByteBuffer = preProcessImage(bitmap);
prediction.setText("Processing");
String pred1 = stackedModel(imageByteBuffer);
//String pred2 = aiModel(bitmap);
prediction.setText(String.format("%s", pred1));
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
public ByteBuffer preProcessImage(Bitmap imgBitmap){
int width = imgBitmap.getWidth();
int height = imgBitmap.getHeight();
int newHeight = 224;
int newWidth = 224;
float scaleWidth = ((float) newWidth) / width;
float scaleHeight = ((float) newHeight) / height;
// Create a matrix for the manipulation
Matrix matrix = new Matrix();
// Resize the bit map
matrix.postScale(scaleWidth, scaleHeight);
// Recreate the new Bitmap
Bitmap resizedBitmap = Bitmap.createBitmap(imgBitmap, 0, 0, width, height, matrix, false);
imageView.setImageBitmap(resizedBitmap);
// Initialization code
// Create an ImageProcessor with all ops required. For more ops, please
// refer to the ImageProcessor Architecture section in this README.
ImageProcessor imageProcessor = new ImageProcessor.Builder()
.add(new ResizeOp(224, 224, ResizeOp.ResizeMethod.BILINEAR)).add(new DequantizeOp(0, 1/255.0f)).build();
// Create a TensorImage object. This creates the tensor of the corresponding
// tensor type (uint8 in this case) that the TensorFlow Lite interpreter needs.
TensorImage tensorImage = new TensorImage(DataType.FLOAT32);
// Preprocess the image
tensorImage.load(resizedBitmap);
tensorImage = imageProcessor.process(tensorImage);
ByteBuffer imageBuffer = tensorImage.getBuffer();
return imageBuffer;
}
public String stackedModel(ByteBuffer byteBuffer){
String debu1 = Float.toString(byteBuffer.getFloat(100));
String debu2 = Float.toString(byteBuffer.getFloat(200));
String debu3 = Float.toString(byteBuffer.getFloat(300));
String debu4 = Float.toString(byteBuffer.getFloat(400));
String debu = "("+debu1+","+ debu2 +","+ debu3+","+ debu4+")";
debuGlobal += debu;
try {
Stacked model = Stacked.newInstance(this);
// Creates inputs for reference.
TensorBuffer inputFeature0 = TensorBuffer.createFixedSize(new int[]{1, 224, 224, 3}, DataType.FLOAT32);
inputFeature0.loadBuffer(byteBuffer);
// Runs model inference and gets result.
Stacked.Outputs outputs = model.process(inputFeature0);
TensorBuffer outputFeature0 = outputs.getOutputFeature0AsTensorBuffer();
float [] out = outputFeature0.getFloatArray();
int leng = out.length;
String pred = String.format("Probability: %s", Double.toString(out[0] / 255.0));
// Releases model resources if no longer used.
model.close();
return pred+"::"+debuGlobal;
} catch (IOException e) {
// TODO Handle the exception
}
return "errored";
}
public String aiModel(Bitmap imgBitmap){
int width = imgBitmap.getWidth();
int height = imgBitmap.getHeight();
int newHeight = 150;
int newWidth = 150;
float scaleWidth = ((float) newWidth) / width;
float scaleHeight = ((float) newHeight) / height;
// Create a matrix for the manipulation
Matrix matrix = new Matrix();
// Resize the bit map
matrix.postScale(scaleWidth, scaleHeight);
// Recreate the new Bitmap
Bitmap resizedBitmap = Bitmap.createBitmap(imgBitmap, 0, 0, width, height, matrix, false);
imageView.setImageBitmap(resizedBitmap);
// Initialization code
// Create an ImageProcessor with all ops required. For more ops, please
// refer to the ImageProcessor Architecture section in this README.
ImageProcessor imageProcessor = new ImageProcessor.Builder()
.add(new ResizeOp(150, 150, ResizeOp.ResizeMethod.BILINEAR))
.build();
// Create a TensorImage object. This creates the tensor of the corresponding
// tensor type (uint8 in this case) that the TensorFlow Lite interpreter needs.
TensorImage tensorImage = new TensorImage(DataType.FLOAT32);
// Preprocess the image
tensorImage.load(resizedBitmap);
tensorImage = imageProcessor.process(tensorImage);
ByteBuffer imageBuffer = tensorImage.getBuffer();
// String debu1 = Float.toString(imageBuffer.getFloat(100));
// String debu2 = Float.toString(imageBuffer.getFloat(200));
// String debu3 = Float.toString(imageBuffer.getFloat(300));
// String debu4 = Float.toString(imageBuffer.getFloat(400));
//
// String debu = debu1 + debu2 + debu3 + debu4;
try {
AiModel model = AiModel.newInstance(this);
// Creates inputs for reference.
TensorBuffer inputFeature0 = TensorBuffer.createFixedSize(new int[]{1, 150, 150, 3}, DataType.FLOAT32);
inputFeature0.loadBuffer(imageBuffer);
// Runs model inference and gets result.
AiModel.Outputs outputs = model.process(inputFeature0);
TensorBuffer outputFeature0 = outputs.getOutputFeature0AsTensorBuffer();
float [] out = outputFeature0.getFloatArray();
int leng = out.length;
String pred = String.format("Probability: %s", Double.toString(out[0] / 255.0));
// Releases model resources if no longer used.
model.close();
return pred;
} catch (IOException e) {
// TODO Handle the exception
}
return "none returned";
}
}
在指责模型之前，我们需要考虑预处理数据(在您的情况下是图像(时可能出现的问题。
所以我建议你检查和测试以下内容：
- 检查图像处理，确保imageByteBuffer已正确初始化，并且当您选择不同的图片时，其内容不同。只需添加一些日志记录，看看这是否属实
- 请确保您在应用程序中正确初始化了TensorFlow模型
- 检查您是否正在从输出数组中的右侧项检索结果
我发现我的张量图像不是量子化的，我试图用/255除以答案。修好后，模型运行良好。
从这个不完整的代码：
ImageProcessor imageProcessor = new ImageProcessor.Builder()
.add(new ResizeOp(150, 150, ResizeOp.ResizeMethod.BILINEAR))
.build();
到完成的代码：
ImageProcessor imageProcessor = new ImageProcessor.Builder()
.add(new ResizeOp(224, 224, ResizeOp.ResizeMethod.BILINEAR)).add(new DequantizeOp(0, 1/255.0f)).build();
由此：
String pred = String.format("Probability: %s", Double.toString(out[0]/ 255.0f));
到此：
String pred = String.format("Probability: %s", Double.toString(out[0]));