标签 网盘 下的文章

利用hdfs搭建网盘–webserver开发

利用hdfs搭建网盘–webserver开发,描述下实现思路:
1、网盘系统中的webserver是用来给用户提供操作界面,接收用户指令,完成文件上传、下载、图片上传、下载和图片预览功能的。
2、其中关于存储相关的功能都是调用hdfs API来完成,而关于文件的相关结构化信息都存储在mysql关系型数据库中;
3、webserver起到的是连接客户和hdfs的作用
4、采用的是SSH框架(Struts2、spring、hibernate)、数据库为mysql,数据模型请参考:利用hdfs搭建网盘–数据模型设计
5、web调用hdfs API的思路是:利用java运行时 运行java jar包,可参考《利用HDFS java API增删改查操作》,例如:

process = Runtime.getRuntime().exec("java -jar /root/hdfs-0.0.1-SNAPSHOT.jar read "+fileInfo.getFilePath()+" /root/file-tmp/"+fileInfo.getFileName());

文件列表页面:

@Action(value = "right", results = { @Result(name = "success", location = "/WEB-INF/npage/right.jsp") })
	public String right() {
		HttpSession session = this.getRequest().getSession();
		User user = (User) session.getAttribute("user");
		if (user != null) {
			page = fileInfoServie.queryUserFileList(0, 20, user.getUserId());
		}
		return "success";
	}

文件下载:

@Action(value = "downloadFile", results = { @Result(name = "success", location = "/right", type = "redirectAction") })
	public String downloadFile(){
		// 获取用户信息
		HttpSession session = this.getRequest().getSession();
		User user = (User) session.getAttribute("user");
		//查出要删除的文件信息
		String fileId = getRequest().getParameter("fileId");
		List<FileInfo> lists = fileInfoServie.findBy("fileId", Long.parseLong(fileId), PropertyFilter.MatchType.EQ);
		FileInfo fileInfo = new FileInfo();
		if(lists != null && lists.size() > 0){
			fileInfo = lists.get(0);
		}
		//路径
		String path = "/root/file-tmp/"+fileInfo.getFileName();
		// 从hdfs取得文件
		Process process;
		try {
			process = Runtime.getRuntime().exec("java -jar /root/hdfs-0.0.1-SNAPSHOT.jar read "+fileInfo.getFilePath()+" /root/file-tmp/"+fileInfo.getFileName());
			InputStreamReader ir = new InputStreamReader(process.getInputStream());
			LineNumberReader input = new LineNumberReader(ir);
			String line;
			while ((line = input.readLine()) != null)
				System.out.println(line);
		} catch (IOException e) {
			e.printStackTrace();
		}
		 try {
            // path是指欲下载的文件的路径。
            File file = new File(path);
            // 取得文件名。
            String filename = file.getName();
            // 取得文件的后缀名。
            String ext = filename.substring(filename.lastIndexOf(".") + 1).toUpperCase();
            // 以流的形式下载文件。
            InputStream fis = new BufferedInputStream(new FileInputStream(path));
            byte[] buffer = new byte[fis.available()];
            fis.read(buffer);
            fis.close();
            // 清空response
            this.getResponse().reset();
            // 设置response的Header
            this.getResponse().addHeader("Content-Disposition", "attachment;filename=" + new String(filename.getBytes()));
            this.getResponse().addHeader("Content-Length", "" + file.length());
            OutputStream toClient = new BufferedOutputStream(this.getResponse().getOutputStream());
            this.getResponse().setContentType("application/octet-stream");
            toClient.write(buffer);
            toClient.flush();
            toClient.close();
	        } catch (IOException ex) {
	            ex.printStackTrace();
	        }
		return null;
	}

删除文件:

@Action(value = "deleteFile", results = { @Result(name = "success", location = "/right", type = "redirectAction") })
	public String deleteFile(){
		// 获取用户信息
		HttpSession session = this.getRequest().getSession();
		User user = (User) session.getAttribute("user");
		//查出要删除的文件信息
		String fileId = getRequest().getParameter("fileId");
		List<FileInfo> lists = fileInfoServie.findBy("fileId", Long.parseLong(fileId), PropertyFilter.MatchType.EQ);
		FileInfo fileInfo = new FileInfo();
		if(lists != null && lists.size() > 0){
			fileInfo = lists.get(0);
		}
		// 将文件从hadoop集群删除
		Process process;
		try {
			process = Runtime.getRuntime().exec("java -jar /root/hdfs-0.0.1-SNAPSHOT.jar delete "+fileInfo.getFilePath());
			InputStreamReader ir = new InputStreamReader(process.getInputStream());
			LineNumberReader input = new LineNumberReader(ir);
			String line;
			while ((line = input.readLine()) != null)
				System.out.println(line);
		} catch (IOException e) {
			e.printStackTrace();
		}
		//从数据库删除
		fileInfoServie.deleteById(fileInfo.getId());
		return "success";
	}

照片列表:

@Action(value = "goPicPage", results = { @Result(name = "success", location = "/WEB-INF/npage/pic-right.jsp") })
	public String goPicPage() {
		HttpSession session = this.getRequest().getSession();
		User user = (User) session.getAttribute("user");
		if (user != null) {
			page = fileInfoServie.queryPicFileList(0, 20, user.getUserId());
		}
		return "success";
	}

上传文件或者图片:

@Action(value = "uploadFile", results = { @Result(name = "success", location = "/right", type = "redirectAction") })
	public String uploadFile() throws IOException {
		// 获取用户信息
		HttpSession session = this.getRequest().getSession();
		User user = (User) session.getAttribute("user");
		boolean isPic = false;
		if (file_name.indexOf(".jpg") > 0 || file_name.indexOf(".gif") > 0) {
			isPic = true;
		}
		// 接收文件
		String path = "";
		if (null != file_name && !file_name.equals("")) {
			path = fileInfoServie.saveFile(methodFile, file_name);
		}
		String fileName = "";
		String[] strArr = path.split("/");
		if (strArr != null && strArr.length > 0) {
			fileName = strArr[strArr.length - 1];
		}
		// 将文件上传到hadoop集群
		Process process;
		try {
			process = Runtime.getRuntime().exec(
					"java -jar /root/hdfs-0.0.1-SNAPSHOT.jar upload " + path + " hdfs://hadoopm:9000/user/root/upload/"
							+ user.getUserName() + "/" + fileName);
			InputStreamReader ir = new InputStreamReader(process.getInputStream());
			LineNumberReader input = new LineNumberReader(ir);
			String line;
			while ((line = input.readLine()) != null)
				System.out.println(line);
		} catch (IOException e) {
			e.printStackTrace();
		}
		FileInfo fileInfo = new FileInfo();
		fileInfo.setCreateTime(new Date());
		fileInfo.setFileName(this.file_name);
		fileInfo.setFilePath("hdfs://hadoopm:9000/user/root/upload/" + user.getUserName() + "/" + fileName);
		File fileTemp = new File(path);
		fileInfo.setFileSize((fileTemp.length() / 1024) > 1 ? (fileTemp.length() / 1024) : 1);
		//判断是否为图片
		if (isPic) {
			fileInfo.setFileType(1);// 0 :普通文件 1:图片
			String tempPath = "/root/" + user.getUserName() + "/"+ System.currentTimeMillis() + "/" + this.file_name;
			ImageScale.resizeFix(new File(path),new File(tempPath), 250, 250);
			fileInfo.setImg(tempPath);
		} else {
			fileInfo.setFileType(0);
		}
		fileInfo.setUserId(user.getUserId());
		fileInfo.setFpos(null);
		fileInfo.setFileId(System.currentTimeMillis());
		fileInfoServie.save(fileInfo);
		return "success";
	}

获取缩略图:

@Action(value = "getsImg")
	public String getsImg() throws IOException {
		String fileId = getRequest().getParameter("fileId");
		List<FileInfo> lists = fileInfoServie.findBy("fileId", Long.parseLong(fileId), PropertyFilter.MatchType.EQ);
		FileInfo fileInfo = new FileInfo();
		if(lists != null && lists.size() > 0){
			fileInfo = lists.get(0);
		}
		FileInputStream is = new FileInputStream(fileInfo.getImg());
		int i = is.available(); // 得到文件大小
		byte data[] = new byte[i];
		is.read(data); // 读数据
		is.close();
		this.getResponse().setContentType("image/*"); // 设置返回的文件类型
		OutputStream toClient = this.getResponse().getOutputStream(); // 得到向客户端输出二进制数据的对象
		toClient.write(data); // 输出数据
		toClient.close();
		return null;
	}

获取大图:

@Action(value = "getbImg")
	public String getbImg() throws IOException{
		// 获取用户信息
		HttpSession session = this.getRequest().getSession();
		User user = (User) session.getAttribute("user");
		//查出要删除的文件信息
		String fileId = getRequest().getParameter("fileId");
		List<FileInfo> lists = fileInfoServie.findBy("fileId", Long.parseLong(fileId), PropertyFilter.MatchType.EQ);
		FileInfo fileInfo = new FileInfo();
		if(lists != null && lists.size() > 0){
			fileInfo = lists.get(0);
		}
		//路径
		String path = "/root/file-tmp/bpic/"+fileInfo.getFileName();
		// 从hdfs取得文件
		Process process;
		try {
			process = Runtime.getRuntime().exec("java -jar /root/hdfs-0.0.1-SNAPSHOT.jar read "+fileInfo.getFilePath()+" /root/file-tmp/bpic/"+fileInfo.getFileName());
			InputStreamReader ir = new InputStreamReader(process.getInputStream());
			LineNumberReader input = new LineNumberReader(ir);
			String line;
			while ((line = input.readLine()) != null)
				System.out.println(line);
		} catch (IOException e) {
			e.printStackTrace();
		}
		File f = new File(path);
        if (!f.exists()) {
            this.getResponse().sendError(404, "File not found!");
            return null;
        }
        BufferedInputStream br = new BufferedInputStream(new FileInputStream(f));
        byte[] buf = new byte[1024];
        int len = 0;
        this.getResponse().reset(); // 非常重要
        // 在线打开方式
        URL u = new URL("file:///" + path);
        this.getResponse().setContentType(u.openConnection().getContentType());
        this.getResponse().setHeader("Content-Disposition", "inline; filename=" + f.getName());
        // 文件名应该编码成UTF-8
        OutputStream out = this.getResponse().getOutputStream();
        while ((len = br.read(buf)) > 0)
            out.write(buf, 0, len);
        br.close();
        out.close();
		return null;
	}

完整Action类如下:

package org.nbc.storage.file.action;
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.LineNumberReader;
import java.io.OutputStream;
import java.net.URL;
import java.util.Date;
import java.util.List;
import javax.annotation.Resource;
import javax.servlet.http.HttpSession;
import org.apache.struts2.convention.annotation.Action;
import org.apache.struts2.convention.annotation.Result;
import org.nbc.storage.common.action.BaseAction;
import org.nbc.storage.common.util.ImageScale;
import org.nbc.storage.file.model.FileInfo;
import org.nbc.storage.file.service.FileInfoServie;
import org.nbc.storage.user.model.User;
import com.sitech.core.orm.PropertyFilter;
import com.sitech.core.utils.Page;
public class FileAction extends BaseAction {
	private static final long serialVersionUID = 1L;
	private Page page;
	private String file_name;
	private File methodFile;
	/** 下载需要的 */
	private InputStream inputStream;
	private String downloadFileName;
	@Resource
	public FileInfoServie fileInfoServie;
	@Action(value = "files", results = { @Result(name = "success", location = "/WEB-INF/npage/main.jsp") })
	public String userFileList() {
		return "success";
	}
	@Action(value = "top", results = { @Result(name = "success", location = "/WEB-INF/npage/top.jsp") })
	public String top() {
		return "success";
	}
	@Action(value = "left", results = { @Result(name = "success", location = "/WEB-INF/npage/left.jsp") })
	public String left() {
		return "success";
	}
	@Action(value = "right_top", results = { @Result(name = "success", location = "/WEB-INF/npage/right_top.jsp") })
	public String right_top() {
		return "success";
	}
	@Action(value = "right", results = { @Result(name = "success", location = "/WEB-INF/npage/right.jsp") })
	public String right() {
		HttpSession session = this.getRequest().getSession();
		User user = (User) session.getAttribute("user");
		if (user != null) {
			page = fileInfoServie.queryUserFileList(0, 20, user.getUserId());
		}
		return "success";
	}
	/**
	 *
	 * 下载文件.
	 *
	 * @return
	 */
	@Action(value = "downloadFile", results = { @Result(name = "success", location = "/right", type = "redirectAction") })
	public String downloadFile(){
		// 获取用户信息
		HttpSession session = this.getRequest().getSession();
		User user = (User) session.getAttribute("user");
		//查出要删除的文件信息
		String fileId = getRequest().getParameter("fileId");
		List lists = fileInfoServie.findBy("fileId", Long.parseLong(fileId), PropertyFilter.MatchType.EQ);
		FileInfo fileInfo = new FileInfo();
		if(lists != null && lists.size() > 0){
			fileInfo = lists.get(0);
		}
		//路径
		String path = "/root/file-tmp/"+fileInfo.getFileName();
		// 从hdfs取得文件
		Process process;
		try {
			process = Runtime.getRuntime().exec("java -jar /root/hdfs-0.0.1-SNAPSHOT.jar read "+fileInfo.getFilePath()+" /root/file-tmp/"+fileInfo.getFileName());
			InputStreamReader ir = new InputStreamReader(process.getInputStream());
			LineNumberReader input = new LineNumberReader(ir);
			String line;
			while ((line = input.readLine()) != null)
				System.out.println(line);
		} catch (IOException e) {
			e.printStackTrace();
		}
		 try {
            // path是指欲下载的文件的路径。
            File file = new File(path);
            // 取得文件名。
            String filename = file.getName();
            // 取得文件的后缀名。
            String ext = filename.substring(filename.lastIndexOf(".") + 1).toUpperCase();
            // 以流的形式下载文件。
            InputStream fis = new BufferedInputStream(new FileInputStream(path));
            byte[] buffer = new byte[fis.available()];
            fis.read(buffer);
            fis.close();
            // 清空response
            this.getResponse().reset();
            // 设置response的Header
            this.getResponse().addHeader("Content-Disposition", "attachment;filename=" + new String(filename.getBytes()));
            this.getResponse().addHeader("Content-Length", "" + file.length());
            OutputStream toClient = new BufferedOutputStream(this.getResponse().getOutputStream());
            this.getResponse().setContentType("application/octet-stream");
            toClient.write(buffer);
            toClient.flush();
            toClient.close();
	        } catch (IOException ex) {
	            ex.printStackTrace();
	        }
		return null;
	}
	/**
	 *
	 * 删除文件.
	 *
	 * @return
	 */
	@Action(value = "deleteFile", results = { @Result(name = "success", location = "/right", type = "redirectAction") })
	public String deleteFile(){
		// 获取用户信息
		HttpSession session = this.getRequest().getSession();
		User user = (User) session.getAttribute("user");
		//查出要删除的文件信息
		String fileId = getRequest().getParameter("fileId");
		List lists = fileInfoServie.findBy("fileId", Long.parseLong(fileId), PropertyFilter.MatchType.EQ);
		FileInfo fileInfo = new FileInfo();
		if(lists != null && lists.size() > 0){
			fileInfo = lists.get(0);
		}
		// 将文件从hadoop集群删除
		Process process;
		try {
			process = Runtime.getRuntime().exec("java -jar /root/hdfs-0.0.1-SNAPSHOT.jar delete "+fileInfo.getFilePath());
			InputStreamReader ir = new InputStreamReader(process.getInputStream());
			LineNumberReader input = new LineNumberReader(ir);
			String line;
			while ((line = input.readLine()) != null)
				System.out.println(line);
		} catch (IOException e) {
			e.printStackTrace();
		}
		//从数据库删除
		fileInfoServie.deleteById(fileInfo.getId());
		return "success";
	}
	@Action(value = "goUploadPage", results = { @Result(name = "success", location = "/WEB-INF/npage/upload-right.jsp") })
	public String goUploadPage() {
		return "success";
	}
	@Action(value = "goPicPage", results = { @Result(name = "success", location = "/WEB-INF/npage/pic-right.jsp") })
	public String goPicPage() {
		HttpSession session = this.getRequest().getSession();
		User user = (User) session.getAttribute("user");
		if (user != null) {
			page = fileInfoServie.queryPicFileList(0, 20, user.getUserId());
		}
		return "success";
	}
	/**
	 *
	 * 上传文件或图片.
	 *
	 * @return
	 * @throws IOException
	 */
	@Action(value = "uploadFile", results = { @Result(name = "success", location = "/right", type = "redirectAction") })
	public String uploadFile() throws IOException {
		// 获取用户信息
		HttpSession session = this.getRequest().getSession();
		User user = (User) session.getAttribute("user");
		boolean isPic = false;
		if (file_name.indexOf(".jpg") > 0 || file_name.indexOf(".gif") > 0) {
			isPic = true;
		}
		// 接收文件
		String path = "";
		if (null != file_name && !file_name.equals("")) {
			path = fileInfoServie.saveFile(methodFile, file_name);
		}
		String fileName = "";
		String[] strArr = path.split("/");
		if (strArr != null && strArr.length > 0) {
			fileName = strArr[strArr.length - 1];
		}
		// 将文件上传到hadoop集群
		Process process;
		try {
			process = Runtime.getRuntime().exec(
					"java -jar /root/hdfs-0.0.1-SNAPSHOT.jar upload " + path + " hdfs://hadoopm:9000/user/root/upload/"
							+ user.getUserName() + "/" + fileName);
			InputStreamReader ir = new InputStreamReader(process.getInputStream());
			LineNumberReader input = new LineNumberReader(ir);
			String line;
			while ((line = input.readLine()) != null)
				System.out.println(line);
		} catch (IOException e) {
			e.printStackTrace();
		}
		FileInfo fileInfo = new FileInfo();
		fileInfo.setCreateTime(new Date());
		fileInfo.setFileName(this.file_name);
		fileInfo.setFilePath("hdfs://hadoopm:9000/user/root/upload/" + user.getUserName() + "/" + fileName);
		File fileTemp = new File(path);
		fileInfo.setFileSize((fileTemp.length() / 1024) > 1 ? (fileTemp.length() / 1024) : 1);
		//判断是否为图片
		if (isPic) {
			fileInfo.setFileType(1);// 0 :普通文件 1:图片
			String tempPath = "/root/" + user.getUserName() + "/"+ System.currentTimeMillis() + "/" + this.file_name;
			ImageScale.resizeFix(new File(path),new File(tempPath), 250, 250);
			fileInfo.setImg(tempPath);
		} else {
			fileInfo.setFileType(0);
		}
		fileInfo.setUserId(user.getUserId());
		fileInfo.setFpos(null);
		fileInfo.setFileId(System.currentTimeMillis());
		fileInfoServie.save(fileInfo);
		return "success";
	}
	/**
	 *
	 * 获取缩略图.
	 *
	 * @return
	 * @throws IOException
	 */
	@Action(value = "getsImg")
	public String getsImg() throws IOException {
		String fileId = getRequest().getParameter("fileId");
		List lists = fileInfoServie.findBy("fileId", Long.parseLong(fileId), PropertyFilter.MatchType.EQ);
		FileInfo fileInfo = new FileInfo();
		if(lists != null && lists.size() > 0){
			fileInfo = lists.get(0);
		}
		FileInputStream is = new FileInputStream(fileInfo.getImg());
		int i = is.available(); // 得到文件大小
		byte data[] = new byte[i];
		is.read(data); // 读数据
		is.close();
		this.getResponse().setContentType("image/*"); // 设置返回的文件类型
		OutputStream toClient = this.getResponse().getOutputStream(); // 得到向客户端输出二进制数据的对象
		toClient.write(data); // 输出数据
		toClient.close();
		return null;
	}
	/**
	 *
	 * 获取大图.
	 *
	 * @return
	 * @throws IOException
	 */
	@Action(value = "getbImg")
	public String getbImg() throws IOException{
		// 获取用户信息
		HttpSession session = this.getRequest().getSession();
		User user = (User) session.getAttribute("user");
		//查出要删除的文件信息
		String fileId = getRequest().getParameter("fileId");
		List lists = fileInfoServie.findBy("fileId", Long.parseLong(fileId), PropertyFilter.MatchType.EQ);
		FileInfo fileInfo = new FileInfo();
		if(lists != null && lists.size() > 0){
			fileInfo = lists.get(0);
		}
		//路径
		String path = "/root/file-tmp/bpic/"+fileInfo.getFileName();
		// 从hdfs取得文件
		Process process;
		try {
			process = Runtime.getRuntime().exec("java -jar /root/hdfs-0.0.1-SNAPSHOT.jar read "+fileInfo.getFilePath()+" /root/file-tmp/bpic/"+fileInfo.getFileName());
			InputStreamReader ir = new InputStreamReader(process.getInputStream());
			LineNumberReader input = new LineNumberReader(ir);
			String line;
			while ((line = input.readLine()) != null)
				System.out.println(line);
		} catch (IOException e) {
			e.printStackTrace();
		}
		File f = new File(path);
        if (!f.exists()) {
            this.getResponse().sendError(404, "File not found!");
            return null;
        }
        BufferedInputStream br = new BufferedInputStream(new FileInputStream(f));
        byte[] buf = new byte[1024];
        int len = 0;
        this.getResponse().reset(); // 非常重要
        // 在线打开方式
        URL u = new URL("file:///" + path);
        this.getResponse().setContentType(u.openConnection().getContentType());
        this.getResponse().setHeader("Content-Disposition", "inline; filename=" + f.getName());
        // 文件名应该编码成UTF-8
        OutputStream out = this.getResponse().getOutputStream();
        while ((len = br.read(buf)) > 0)
            out.write(buf, 0, len);
        br.close();
        out.close();
		return null;
	}
	public FileInfoServie getFileInfoServie() {
		return fileInfoServie;
	}
	public void setFileInfoServie(FileInfoServie fileInfoServie) {
		this.fileInfoServie = fileInfoServie;
	}
	public Page getPage() {
		return page;
	}
	public void setPage(Page page) {
		this.page = page;
	}
	public String getFile_name() {
		return file_name;
	}
	public void setFile_name(String file_name) {
		this.file_name = file_name;
	}
	public File getMethodFile() {
		return methodFile;
	}
	public void setMethodFile(File methodFile) {
		this.methodFile = methodFile;
	}
	public String getDownloadFileName() {
		return downloadFileName;
	}
	public void setDownloadFileName(String downloadFileName) {
		this.downloadFileName = downloadFileName;
	}
	public void setInputStream(InputStream inputStream) {
		this.inputStream = inputStream;
	}
	@Action(value = "filelist", results = { @Result(name = "success", location = "/WEB-INF/npage/filelist.jsp") })
	public String filelist() {
//		HttpSession session = this.getRequest().getSession();
//		User user = (User) session.getAttribute("user");
//		if (user != null) {
			page = fileInfoServie.queryUserFileList(0, 20, 1L);
//		}
		return "success";
	}
	@Action(value = "goUploadPageCC", results = { @Result(name = "success", location = "/WEB-INF/npage/goUploadPageCC.jsp") })
	public String goUploadPageCC() {
		return "success";
	}
	@Action(value = "uploadFileCC", results = { @Result(name = "success", location = "/filelist", type = "redirectAction") })
	public String uploadFileCC() throws IOException {
		// 获取用户信息
		HttpSession session = this.getRequest().getSession();
		User user = (User) session.getAttribute("user");
		boolean isPic = false;
		if (file_name.indexOf(".jpg") > 0 || file_name.indexOf(".gif") > 0) {
			isPic = true;
		}
		// 接收文件
		String path = "";
		if (null != file_name && !file_name.equals("")) {
			path = fileInfoServie.saveFile(methodFile, file_name);
		}
		String fileName = "";
		String[] strArr = path.split("/");
		if (strArr != null && strArr.length > 0) {
			fileName = strArr[strArr.length - 1];
		}
		// 将文件上传到hadoop集群
		Process process;
		try {
			process = Runtime.getRuntime().exec(
					"java -jar /root/hdfs-0.0.1-SNAPSHOT.jar upload " + path + " hdfs://hadoopm:9000/user/root/upload/"
							+ "lisn" + "/" + fileName);
			InputStreamReader ir = new InputStreamReader(process.getInputStream());
			LineNumberReader input = new LineNumberReader(ir);
			String line;
			while ((line = input.readLine()) != null)
				System.out.println(line);
		} catch (IOException e) {
			e.printStackTrace();
		}
		FileInfo fileInfo = new FileInfo();
		fileInfo.setCreateTime(new Date());
		fileInfo.setFileName(this.file_name);
		fileInfo.setFilePath("hdfs://hadoopm:9000/user/root/upload/" +"lisn" + "/" + fileName);
		File fileTemp = new File(path);
		fileInfo.setFileSize((fileTemp.length() / 1024) > 1 ? (fileTemp.length() / 1024) : 1);
		//判断是否为图片
		if (isPic) {
			fileInfo.setFileType(1);// 0 :普通文件 1:图片
			String tempPath = "/root/" + "lisn" + "/"+ System.currentTimeMillis() + "/" + this.file_name;
			ImageScale.resizeFix(new File(path),new File(tempPath), 250, 250);
			fileInfo.setImg(tempPath);
		} else {
			fileInfo.setFileType(0);
		}
		fileInfo.setUserId(1L);
		fileInfo.setFpos(null);
		fileInfo.setFileId(System.currentTimeMillis());
		fileInfoServie.save(fileInfo);
		return "success";
	}
}

完整services:

package org.nbc.storage.file.service.impl;
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import javax.annotation.Resource;
import org.nbc.storage.file.model.FileInfo;
import org.nbc.storage.file.service.FileInfoServie;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import com.sitech.core.orm.BaseDAO;
import com.sitech.core.service.impl.BaseServiceImpl;
import com.sitech.core.utils.Page;
@Service("fileInfoServie")
@Transactional
public class FileInfoServieImpl extends BaseServiceImpl<FileInfo, Long> implements FileInfoServie {
	private static int BUFFER_SIZE = 16 * 1024;
	@Resource(name = "fileInfoDAO")
	public void setBaseDAO(BaseDAO<FileInfo, Long> baseDAO) {
		super.setBaseDAO(baseDAO);
	}
	@Override
	public Page<FileInfo> queryUserFileList(int pageNo,
			int pageSize, Long userId) {
		String hql = " from FileInfo where userId = ? order by createTime desc ";
		Page<FileInfo> page = new Page<FileInfo>(pageSize);
		page.setPageNo(pageNo);
		return this.getBaseDAO().findPage(page, hql, userId);
	}
	public long copyFile(File src, File dest) throws Exception {
		BufferedInputStream in = null;
		BufferedOutputStream out = null;
		byte[] buffer = new byte[BUFFER_SIZE];
		long total = 0;
		try {
			in = new BufferedInputStream(new FileInputStream(src), BUFFER_SIZE);
			out = new BufferedOutputStream(new FileOutputStream(dest),
					BUFFER_SIZE);
			int curSize = in.read(buffer);
			while (curSize > 0) {
				total += curSize;
				out.write(buffer, 0, curSize);
				curSize = in.read(buffer);
			}
		} catch (Exception e) {
		} finally {
			try {
				if (in != null) {
					in.close();
					in = null;
				}
				if (out != null) {
					out.close();
					out = null;
				}
			} catch (Exception e) {
				e.printStackTrace();
			}
		}
		return total;
	}
	public String saveFile(File methodFile,String file_name){
		String savePath = "/root/lisn/";
		String middlePath = "";
		String separator = "/";
		String [] strArr = file_name.split("\.");
		if(strArr != null && strArr.length > 0){
			file_name = strArr[strArr.length-1];
		}
		//创建filename
		file_name = System.currentTimeMillis() + "."+file_name;
		File f = new File(savePath);
		if (!f.exists()) {
			f.mkdirs();
		}
		File file = new File(savePath+file_name);
		try {
			file.createNewFile();
			long total = copyFile(methodFile, file);
		} catch (Exception e) {
			e.printStackTrace();
		}
		return savePath+file_name;
	}
	@Override
	public Page<FileInfo> queryPicFileList(int pageNo, int pageSize, Long userId) {
		String hql = " from FileInfo where userId = ? and fileType = 1 order by createTime desc ";
		Page<FileInfo> page = new Page<FileInfo>(pageSize);
		page.setPageNo(pageNo);
		return this.getBaseDAO().findPage(page, hql, userId);
	}
}

展示图片预览页面:

<%@ page language="java" import="java.util.*" pageEncoding="UTF-8"%>
<%
String path = request.getContextPath();
String basePath = request.getScheme()+"://"+request.getServerName()+":"+request.getServerPort()+path+"/";
%>
<%@ taglib prefix="s" uri="/struts-tags" %>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>无标题文档</title>
<style type="text/css">
<!--
.STYLE1 {font-size: 12px}
.STYLE3 {color: #707070; font-size: 12px; }
.STYLE5 {color: #0a6e0c; font-size: 12px; }
body {
	margin-top: 0px;
	margin-bottom: 0px;
}
.STYLE7 {font-size: 12}
-->
</style>
</head>
<body>
<table width="100%" border="0" cellspacing="0" cellpadding="0">
  <tr>
    <td height="30"><table width="100%" border="0" cellspacing="0" cellpadding="0">
      <tr>
        <td>&nbsp;</td>
        <td style="padding-right:10px;"><div align="right">
          <table border="0" align="right" cellpadding="0" cellspacing="0">
            <tr>
              <td width="60"><table width="87%" border="0" cellpadding="0" cellspacing="0">
                  <tr>
                   <!--  <td class="STYLE1"><div align="center">
                        <input type="checkbox" name="checkbox62" value="checkbox" />
                    </div></td>-->
                    <td class="STYLE1" onclick="window.location.href='<%=request.getContextPath() %>/wangpan-web/goPicPage'"><div align="center" style="cursor:pointer;">图片列表</div></td>
                  </tr>
              </table></td>
              <td width="60"><table width="90%" border="0" cellpadding="0" cellspacing="0">
                  <tr>
                  <!--   <td class="STYLE1"><div align="center"><img src="<%=request.getContextPath() %>/images/001.gif" width="14" height="14" /></div></td> -->
                    <td class="STYLE1" onclick="window.location.href='<%=request.getContextPath() %>/wangpan-web/goUploadPage'"><div align="center" style="cursor:pointer;">上传文件</div></td>
                  </tr>
              </table></td>
              <td width="60"><table width="90%" border="0" cellpadding="0" cellspacing="0">
                  <tr>
                   <!--  <td class="STYLE1"><div align="center"><img src="<%=request.getContextPath() %>/images/114.gif" width="14" height="14" /></div></td> -->
                    <td class="STYLE1" onclick="window.location.href='<%=request.getContextPath() %>/wangpan-web/right'"><div align="center" style="cursor:pointer;">文件列表</div></td>
                  </tr>
              </table></td>
              <!--
              <td width="52"><table width="88%" border="0" cellpadding="0" cellspacing="0">
                  <tr>
                    <td class="STYLE1"><div align="center"><img src="<%=request.getContextPath() %>/images/083.gif" width="14" height="14" /></div></td>
                    <td class="STYLE1"><div align="center">删除</div></td>
                  </tr>
              </table></td>
               -->
            </tr>
          </table>
        </div></td>
      </tr>
    </table></td>
  </tr>
  <tr>
    <td><table width="100%" border="0" cellpadding="0" cellspacing="1" bgcolor="#c9c9c9">
      <tr>
        <td height="22" bgcolor="#FFFFFF" width="50%"><div align="center"><strong><span class="STYLE1">图片预览</span></strong></div></td>
      </tr>
      <s:iterator value="page.result" var="file">
      <tr>
        <td height="22" bgcolor="#FFFFFF" width="50%"><div align="center"><span class="STYLE3" style="cursor:pointer;"><img onclick="window.open('http://192.168.75.142:8080/wangpan-web/getbImg?fileId=<s:property value="fileId" />');" title="点击查看大图" alt="<s:property value="fileName" />" src="http://192.168.75.142:8080/wangpan-web/getsImg?fileId=<s:property value="fileId" />" >  </span></div></td>
      </tr>
      </s:iterator>
    </table></td>
  </tr>
  <tr>
    <td height="35"><table width="100%" border="0" cellspacing="0" cellpadding="0">
      <tr>
        <td width="25%" height="29" nowrap="nowrap"><table width="342" border="0" cellspacing="0" cellpadding="0">
          <tr>
            <td width="44%" class="STYLE1">当前页:1/2页 每页
              <input name="textfield2" type="text" class="STYLE1" style="height:14px; width:25px;" value="15" size="5" />            </td>
            <td width="14%" class="STYLE1"><img src="<%=request.getContextPath() %>/images/sz.gif" width="43" height="20" /></td>
            <td width="42%" class="STYLE1"><span class="STYLE7">数据总量 15 </span></td>
          </tr>
        </table></td>
        <td width="75%" valign="top" class="STYLE1"><div align="right">
            <table width="352" height="20" border="0" cellpadding="0" cellspacing="0">
              <tr>
                <td width="62" height="22" valign="middle"><div align="right"><img src="<%=request.getContextPath() %>/images/page_first_1.gif" width="48" height="20" /></div></td>
                <td width="50" height="22" valign="middle"><div align="right"><img src="<%=request.getContextPath() %>/images/page_back_1.gif" width="55" height="20" /></div></td>
                <td width="54" height="22" valign="middle"><div align="right"><img src="<%=request.getContextPath() %>/images/page_next.gif" width="58" height="20" /></div></td>
                <td width="49" height="22" valign="middle"><div align="right"><img src="<%=request.getContextPath() %>/images/page_last.gif" width="52" height="20" /></div></td>
                <td width="59" height="22" valign="middle"><div align="right">转到第</div></td>
                <td width="25" height="22" valign="middle"><span class="STYLE7">
                  <input name="textfield" type="text" class="STYLE1" style="height:10px; width:25px;" size="5" />
                </span></td>
                <td width="23" height="22" valign="middle">页</td>
                <td width="30" height="22" valign="middle"><img src="<%=request.getContextPath() %>/images/go.gif" width="26" height="20" /></td>
              </tr>
            </table>
        </div></td>
      </tr>
    </table></td>
  </tr>
</table>
</body>
</html>

文件列表页:

<%@ page language="java" import="java.util.*" pageEncoding="UTF-8"%>
<%
String path = request.getContextPath();
String basePath = request.getScheme()+"://"+request.getServerName()+":"+request.getServerPort()+path+"/";
%>
<%@ taglib prefix="s" uri="/struts-tags" %>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>无标题文档</title>
<style type="text/css">
<!--
.STYLE1 {font-size: 12px}
.STYLE3 {color: #707070; font-size: 12px; }
.STYLE5 {color: #0a6e0c; font-size: 12px; }
body {
	margin-top: 0px;
	margin-bottom: 0px;
}
.STYLE7 {font-size: 12}
-->
</style>
</head>
<body>
<table width="100%" border="0" cellspacing="0" cellpadding="0">
  <tr>
    <td height="30"><table width="100%" border="0" cellspacing="0" cellpadding="0">
      <tr>
        <td>&nbsp;</td>
        <td style="padding-right:10px;"><div align="right">
          <table border="0" align="right" cellpadding="0" cellspacing="0">
            <tr>
              <td width="60"><table width="87%" border="0" cellpadding="0" cellspacing="0">
                  <tr>
                   <!--  <td class="STYLE1"><div align="center">
                        <input type="checkbox" name="checkbox62" value="checkbox" />
                    </div></td>-->
                    <td class="STYLE1" onclick="window.location.href='<%=request.getContextPath() %>/wangpan-web/goPicPage'"><div align="center" style="cursor:pointer;">图片列表</div></td>
                  </tr>
              </table></td>
              <td width="60"><table width="90%" border="0" cellpadding="0" cellspacing="0">
                  <tr>
                  <!--   <td class="STYLE1"><div align="center"><img src="<%=request.getContextPath() %>/images/001.gif" width="14" height="14" /></div></td> -->
                    <td class="STYLE1" onclick="window.location.href='<%=request.getContextPath() %>/wangpan-web/goUploadPage'"><div align="center" style="cursor:pointer;">上传文件</div></td>
                  </tr>
              </table></td>
              <td width="60"><table width="90%" border="0" cellpadding="0" cellspacing="0">
                  <tr>
                   <!--  <td class="STYLE1"><div align="center"><img src="<%=request.getContextPath() %>/images/114.gif" width="14" height="14" /></div></td> -->
                    <td class="STYLE1" onclick="window.location.href='<%=request.getContextPath() %>/wangpan-web/right'"><div align="center" style="cursor:pointer;">文件列表</div></td>
                  </tr>
              </table></td>
              <!--
              <td width="52"><table width="88%" border="0" cellpadding="0" cellspacing="0">
                  <tr>
                    <td class="STYLE1"><div align="center"><img src="<%=request.getContextPath() %>/images/083.gif" width="14" height="14" /></div></td>
                    <td class="STYLE1"><div align="center">删除</div></td>
                  </tr>
              </table></td>
               -->
            </tr>
          </table>
        </div></td>
      </tr>
    </table></td>
  </tr>
  <tr>
    <td><table width="100%" border="0" cellpadding="0" cellspacing="1" bgcolor="#c9c9c9">
      <tr>
        <td height="22" bgcolor="#FFFFFF" width="50%"><div align="center"><strong><span class="STYLE1">文件名</span></strong></div></td>
        <td height="22" bgcolor="#FFFFFF"><div align="center"><strong><span class="STYLE1">大小</span></strong></div></td>
        <td height="22" bgcolor="#FFFFFF"><div align="center"><strong><span class="STYLE1">上传日期</span></strong></div></td>
        <td height="22" bgcolor="#FFFFFF"><div align="center"><strong><span class="STYLE1">操作</span></strong></div></td>
      </tr>
      <s:iterator value="page.result" var="file">
      <tr>
        <td height="22" bgcolor="#FFFFFF" width="50%"><div align="center"><span class="STYLE3"><s:property value="fileName" /></span></div></td>
        <td height="22" bgcolor="#FFFFFF"><div align="center"><span class="STYLE3"><s:property value="fileSize" />&nbsp;KB</span></div></td>
        <td height="22" bgcolor="#FFFFFF"><div align="center"><span class="STYLE3"><s:date name="createTime" format="yyyy-MM-dd HH:mm" /></span></div></td>
        <td height="22" bgcolor="#FFFFFF">
        	<div align="center">
        		<span style="cursor:pointer;" class="STYLE3" onclick="window.location.href='<%=request.getContextPath() %>/wangpan-web/deleteFile?fileId=<s:property value="fileId" />'">删除</span>
        		<span style="cursor:pointer;" class="STYLE3" onclick="window.open('<%=request.getContextPath() %>/wangpan-web/downloadFile?fileId=<s:property value="fileId" />');">下载</span>
        		</div>
        </td>
      </tr>
      </s:iterator>
    </table></td>
  </tr>
  <tr>
    <td height="35"><table width="100%" border="0" cellspacing="0" cellpadding="0">
      <tr>
        <td width="25%" height="29" nowrap="nowrap"><table width="342" border="0" cellspacing="0" cellpadding="0">
          <tr>
            <td width="44%" class="STYLE1">当前页:1/2页 每页
              <input name="textfield2" type="text" class="STYLE1" style="height:14px; width:25px;" value="15" size="5" />            </td>
            <td width="14%" class="STYLE1"><img src="<%=request.getContextPath() %>/images/sz.gif" width="43" height="20" /></td>
            <td width="42%" class="STYLE1"><span class="STYLE7">数据总量 15 </span></td>
          </tr>
        </table></td>
        <td width="75%" valign="top" class="STYLE1"><div align="right">
            <table width="352" height="20" border="0" cellpadding="0" cellspacing="0">
              <tr>
                <td width="62" height="22" valign="middle"><div align="right"><img src="<%=request.getContextPath() %>/images/page_first_1.gif" width="48" height="20" /></div></td>
                <td width="50" height="22" valign="middle"><div align="right"><img src="<%=request.getContextPath() %>/images/page_back_1.gif" width="55" height="20" /></div></td>
                <td width="54" height="22" valign="middle"><div align="right"><img src="<%=request.getContextPath() %>/images/page_next.gif" width="58" height="20" /></div></td>
                <td width="49" height="22" valign="middle"><div align="right"><img src="<%=request.getContextPath() %>/images/page_last.gif" width="52" height="20" /></div></td>
                <td width="59" height="22" valign="middle"><div align="right">转到第</div></td>
                <td width="25" height="22" valign="middle"><span class="STYLE7">
                  <input name="textfield" type="text" class="STYLE1" style="height:10px; width:25px;" size="5" />
                </span></td>
                <td width="23" height="22" valign="middle">页</td>
                <td width="30" height="22" valign="middle"><img src="<%=request.getContextPath() %>/images/go.gif" width="26" height="20" /></td>
              </tr>
            </table>
        </div></td>
      </tr>
    </table></td>
  </tr>
</table>
</body>
</html>

利用hdfs搭建网盘–数据模型设计

先阐述下利用hdfs搭建网盘的思路:
(1)、首先要搭建hadoop集群,确保该集群正常运行
(2)、通过API访问文件到存储在hdfs中的文件,能对文件进行增删改查
(3)、文件的其他结构化信息,,比如:文件名称,上传时间,所属用户、文件类型等信息,需要存储在数据库里,我们使用mysql
(4)、用户需要通过操作界面来访问网盘系统,而不是直接操作hdfs,这里采用java、struts2框架来实现web端开发
(5)、有用户系统,存储用户相关信息,另外hdfs中文件存放的路径也和用户有直接关系
网盘系统的截图:http://pan.baidu.com/share/link?shareid=3253971941&uk=772112791
第一点已经在《Hadoop集群搭建详细简明教程》里详细写明步骤了,再次就不再提了
第二点已经在《利用HDFS java API增删改查操作》里详细阐述了
这篇文章先阐述下第三、第五点
网盘数据模型的设计,以及mysql在linux下的安装
网盘数据模型的设计:

file_info

file_info


user

user


tables

tables


目前只用到这file_ifno,user这两个表
mysql在linux:
我是在《Hadoop集群搭建详细简明教程》中搭建好的hadoopm主机上,通过yum安装的mysql,并设置mysql密码

yum install mysql
yum install mysql-server
yum install mysql-devel
chgrp -R mysql /var/lib/mysql
chmod -R 770 /var/lib/mysql
service mysqld start
mysql
SET PASSWORD FOR 'root'@'localhost' = PASSWORD('secret_password');

PS:设置密码的时候可能会报错:ERROR 1372 (HY000): Password hash should be a 41-digit hexadecimal number
可通过这个命令:select password(‘root’);得知root的 41-digit是什么
设置mysql开机启动:

chkconfig --levels 345 mysqld on

相关参考资料:
1、linux下安装mysql
2、设置用户
http://www.hackbase.com/tech/2011-09-09/65234.html
yum install mysql
http://www.cnblogs.com/xiaochaohuashengmi/archive/2011/10/16/2214272.html
3、远程连接
http://www.cnblogs.com/smallstone/archive/2010/04/29/1723838.html
4、创建数据库–奇怪的问题:必须得使用特别奇怪的引号,否则不生效
http://tdcq.iteye.com/blog/363955
http://blog.sina.com.cn/s/blog_5dc960cd0100ea2h.html
5、在数据库上创建表
http://www.51cto.com/html/2005/1129/12524.htm
6、mysql中文乱码
http://www.2cto.com/database/201108/101151.html
7、修改表结构
http://database.51cto.com/art/201005/201148.htm
8、2007-07-20 10:59 在mysql中查询一个数据库中的所有表
http://hi.baidu.com/aigyoo/item/d56a8c01dcb50c10cd34eacf
9、mysql查看表结构命令
http://www.blogjava.net/etlan/archive/2007/07/12/129794.html